565 citations · 991 across the 20 of their papers we have counts for
23 papers
Relaxed Recursive Transformers: Effective Parameter Sharing with Layer-wise LoRA
Sangmin Bae, Adam Fisch, Hrayr Harutyunyan +3
Large language models (LLMs) are expensive to deploy. Parameter sharing offers a possible path towards reducing their size and cost, but its effectiveness in modern LLMs remains fa…
Block Transformer: Global-to-Local Language Modeling for Fast Inference
Namgyu Ho, Sangmin Bae, Taehyeon Kim +6
We introduce the Block Transformer which adopts hierarchical global-to-local modeling to autoregressive transformers to mitigate the inference bottlenecks associated with self-atte…
Risk-Controlling Model Selection via Guided Bayesian Optimization
Bracha Laufer-Goldshtein, Adam Fisch, Regina Barzilay +1
Adjustable hyperparameters of machine learning models typically impact various key trade-offs such as accuracy, fairness, robustness, or inference cost. Our goal in this paper is t…
Towards Robust and Efficient Continual Language Learning
Adam Fisch, Amal Rannen-Triki, Razvan Pascanu +4
As the application space of language models continues to evolve, a natural question to ask is how we can quickly adapt models to new tasks. We approach this classic question from a…
Conformal Language Modeling
Victor Quach, Adam Fisch, Tal Schuster +4
We propose a novel approach to conformal prediction for generative language models (LMs). Standard conformal prediction produces prediction sets -- in place of single predictions -…
Efficiently Controlling Multiple Risks with Pareto Testing
Bracha Laufer-Goldshtein, Adam Fisch, Regina Barzilay +1
Machine learning applications frequently come with multiple diverse objectives and constraints that can change over time. Accordingly, trained models can be tuned with sets of hype…
Calibrated Selective Classification
Adam Fisch, Tommi Jaakkola, Regina Barzilay
Selective classification allows models to abstain from making predictions (e.g., say "I don't know") when in doubt in order to obtain better effective accuracy. While typical selec…
Conformal Risk Control
Anastasios N. Angelopoulos, Stephen Bates, Adam Fisch +2
We extend conformal prediction to control the expected value of any monotone loss function. The algorithm generalizes split conformal prediction together with its coverage guarante…
Confident Adaptive Language Modeling
Tal Schuster, Adam Fisch, Jai Gupta +5
Recent advances in Transformer-based large language models (LLMs) have led to significant performance improvements across many tasks. These gains come with a drastic increase in th…
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao +448
Language models demonstrate both quantitative improvement and new qualitative capabilities with increasing scale. Despite their potentially transformative impact, these new capabil…
Conformal Prediction Sets with Limited False Positives
Adam Fisch, Tal Schuster, Tommi Jaakkola +1
We develop a new approach to multi-label conformal prediction in which we aim to output a precise set of promising prediction candidates with a bounded number of incorrect answers.…
Consistent Accelerated Inference via Confident Adaptive Transformers
Tal Schuster, Adam Fisch, Tommi Jaakkola +1
We develop a novel approach for confidently accelerating inference in the large and expensive multilayer Transformers that are now ubiquitous in natural language processing (NLP).…
Get Your Vitamin C! Robust Fact Verification with Contrastive Evidence
Tal Schuster, Adam Fisch, Regina Barzilay
Typical fact verification models use retrieved written evidence to verify claims. Evidence sources, however, often change over time as more information is gathered and revised. In…
Few-shot Conformal Prediction with Auxiliary Tasks
Adam Fisch, Tal Schuster, Tommi Jaakkola +1
We develop a novel approach to conformal prediction when the target task has limited data available for training. Conformal prediction identifies a small set of promising output ca…
Making Pre-trained Language Models Better Few-shot Learners
Tianyu Gao, Adam Fisch, Danqi Chen
The recent GPT-3 model (Brown et al., 2020) achieves remarkable few-shot performance solely by leveraging a natural-language prompt and a few task demonstrations as input context.…
CapWAP: Captioning with a Purpose
Adam Fisch, Kenton Lee, Ming-Wei Chang +2
The traditional image captioning task uses generic reference captions to provide textual information about images. Different user populations, however, will care about different vi…
Efficient Conformal Prediction via Cascaded Inference with Expanded Admission
Adam Fisch, Tal Schuster, Tommi Jaakkola +1
In this paper, we present a novel approach for conformal prediction (CP), in which we aim to identify a set of promising prediction candidates -- in place of a single prediction. T…
MRQA 2019 Shared Task: Evaluating Generalization in Reading Comprehension
Adam Fisch, Alon Talmor, Robin Jia +3
We present the results of the Machine Reading for Question Answering (MRQA) 2019 shared task on evaluating the generalization capabilities of reading comprehension systems. In this…
Working Hard or Hardly Working: Challenges of Integrating Typology into Neural Dependency Parsers
Adam Fisch, Jiang Guo, Regina Barzilay
This paper explores the task of leveraging typology in the context of cross-lingual dependency parsing. While this linguistic information has shown great promise in pre-neural pars…
StarSpace: Embed All The Things!
Ledell Wu, Adam Fisch, Sumit Chopra +3
We present StarSpace, a general-purpose neural embedding model that can solve a wide variety of problems: labeling tasks such as text classification, ranking tasks such as informat…
ParlAI: A Dialog Research Software Platform
Alexander H. Miller, Will Feng, Adam Fisch +5
We introduce ParlAI (pronounced "par-lay"), an open-source software platform for dialog research implemented in Python, available at http://parl.ai. Its goal is to provide a unifie…
Reading Wikipedia to Answer Open-Domain Questions
Danqi Chen, Adam Fisch, Jason Weston +1
This paper proposes to tackle open- domain question answering using Wikipedia as the unique knowledge source: the answer to any factoid question is a text span in a Wikipedia artic…
Key-Value Memory Networks for Directly Reading Documents
Alexander Miller, Adam Fisch, Jesse Dodge +3
Directly reading documents and being able to answer questions from them is an unsolved challenge. To avoid its inherent difficulty, question answering (QA) has been directed toward…