565 citations · 991 across the 20 of their papers we have counts for
16 papers · 1 filter
Relaxed Recursive Transformers: Effective Parameter Sharing with Layer-wise LoRA
Sangmin Bae, Adam Fisch, Hrayr Harutyunyan +3
Large language models (LLMs) are expensive to deploy. Parameter sharing offers a possible path towards reducing their size and cost, but its effectiveness in modern LLMs remains fa…
Block Transformer: Global-to-Local Language Modeling for Fast Inference
Namgyu Ho, Sangmin Bae, Taehyeon Kim +6
We introduce the Block Transformer which adopts hierarchical global-to-local modeling to autoregressive transformers to mitigate the inference bottlenecks associated with self-atte…
Towards Robust and Efficient Continual Language Learning
Adam Fisch, Amal Rannen-Triki, Razvan Pascanu +4
As the application space of language models continues to evolve, a natural question to ask is how we can quickly adapt models to new tasks. We approach this classic question from a…
Conformal Language Modeling
Victor Quach, Adam Fisch, Tal Schuster +4
We propose a novel approach to conformal prediction for generative language models (LMs). Standard conformal prediction produces prediction sets -- in place of single predictions -…
Confident Adaptive Language Modeling
Tal Schuster, Adam Fisch, Jai Gupta +5
Recent advances in Transformer-based large language models (LLMs) have led to significant performance improvements across many tasks. These gains come with a drastic increase in th…
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao +448
Language models demonstrate both quantitative improvement and new qualitative capabilities with increasing scale. Despite their potentially transformative impact, these new capabil…
Consistent Accelerated Inference via Confident Adaptive Transformers
Tal Schuster, Adam Fisch, Tommi Jaakkola +1
We develop a novel approach for confidently accelerating inference in the large and expensive multilayer Transformers that are now ubiquitous in natural language processing (NLP).…
Get Your Vitamin C! Robust Fact Verification with Contrastive Evidence
Tal Schuster, Adam Fisch, Regina Barzilay
Typical fact verification models use retrieved written evidence to verify claims. Evidence sources, however, often change over time as more information is gathered and revised. In…
Making Pre-trained Language Models Better Few-shot Learners
Tianyu Gao, Adam Fisch, Danqi Chen
The recent GPT-3 model (Brown et al., 2020) achieves remarkable few-shot performance solely by leveraging a natural-language prompt and a few task demonstrations as input context.…
CapWAP: Captioning with a Purpose
Adam Fisch, Kenton Lee, Ming-Wei Chang +2
The traditional image captioning task uses generic reference captions to provide textual information about images. Different user populations, however, will care about different vi…
MRQA 2019 Shared Task: Evaluating Generalization in Reading Comprehension
Adam Fisch, Alon Talmor, Robin Jia +3
We present the results of the Machine Reading for Question Answering (MRQA) 2019 shared task on evaluating the generalization capabilities of reading comprehension systems. In this…
Working Hard or Hardly Working: Challenges of Integrating Typology into Neural Dependency Parsers
Adam Fisch, Jiang Guo, Regina Barzilay
This paper explores the task of leveraging typology in the context of cross-lingual dependency parsing. While this linguistic information has shown great promise in pre-neural pars…
StarSpace: Embed All The Things!
Ledell Wu, Adam Fisch, Sumit Chopra +3
We present StarSpace, a general-purpose neural embedding model that can solve a wide variety of problems: labeling tasks such as text classification, ranking tasks such as informat…
ParlAI: A Dialog Research Software Platform
Alexander H. Miller, Will Feng, Adam Fisch +5
We introduce ParlAI (pronounced "par-lay"), an open-source software platform for dialog research implemented in Python, available at http://parl.ai. Its goal is to provide a unifie…
Reading Wikipedia to Answer Open-Domain Questions
Danqi Chen, Adam Fisch, Jason Weston +1
This paper proposes to tackle open- domain question answering using Wikipedia as the unique knowledge source: the answer to any factoid question is a text span in a Wikipedia artic…
Key-Value Memory Networks for Directly Reading Documents
Alexander Miller, Adam Fisch, Jesse Dodge +3
Directly reading documents and being able to answer questions from them is an unsolved challenge. To avoid its inherent difficulty, question answering (QA) has been directed toward…