2.2k citations · 2.5k across the 23 of their papers we have counts for
5 papers · 1 filter
Knowledge Enhanced Contextual Word Representations
Matthew E. Peters, Mark Neumann, Robert L. Logan +4
Contextual word representations, typically trained on unstructured, unlabeled text, do not contain any explicit grounding to real world entities and are often unable to remember fa…
Shallow Syntax in Deep Water
Swabha Swayamdipta, Matthew Peters, Brendan Roof +2
Shallow syntax provides an approximation of phrase-syntactic structure of sentences; it can be produced with high accuracy, and is computationally cheap to obtain. We investigate t…
Barack's Wife Hillary: Using Knowledge-Graphs for Fact-Aware Language Modeling
Robert L. Logan, Nelson F. Liu, Matthew E. Peters +2
Modeling human language requires the ability to not only generate fluent text but also encode factual knowledge. However, traditional language models are only capable of rememberin…
Linguistic Knowledge and Transferability of Contextual Representations
Nelson F. Liu, Matt Gardner, Yonatan Belinkov +2
Contextual word representations derived from large-scale neural language models are successful across a diverse set of NLP tasks, suggesting that they encode useful and transferabl…
To Tune or Not to Tune? Adapting Pretrained Representations to Diverse Tasks
Matthew E. Peters, Sebastian Ruder, Noah A. Smith
While most previous work has focused on different pretraining objectives and architectures for transfer learning, we ask how to best adapt the pretrained model to a given target ta…