62 citations · 261 across the 9 of their papers we have counts for
4 papers · 1 filter
Generalization through Memorization: Nearest Neighbor Language Models
Urvashi Khandelwal, Omer Levy, Dan Jurafsky +2
We introduce NN-LMs, which extend a pre-trained neural language model (LM) by linearly interpolating it with a -nearest neighbors (NN) model. The nearest neighbors are com…
BAM! Born-Again Multi-Task Networks for Natural Language Understanding
Kevin Clark, Minh-Thang Luong, Urvashi Khandelwal +2
It can be challenging to train multi-task neural networks that outperform or even match their single-task counterparts. To help address this, we propose using knowledge distillatio…
What Does BERT Look At? An Analysis of BERT's Attention
Kevin Clark, Urvashi Khandelwal, Omer Levy +1
Large pre-trained neural networks such as BERT have had great recent success in NLP, motivating a growing body of research investigating what aspects of language they are able to l…
Sample Efficient Text Summarization Using a Single Pre-Trained Transformer
Urvashi Khandelwal, Kevin Clark, Dan Jurafsky +1
Language model (LM) pre-training has resulted in impressive performance and sample efficiency on a variety of language understanding tasks. However, it remains unclear how to best…