139 citations · 373 across the 15 of their papers we have counts for
16 papers
Simfluence: Modeling the Influence of Individual Training Examples by Simulating Training Runs
Kelvin Guu, Albert Webson, Ellie Pavlick +3
Training data attribution (TDA) methods offer to trace a model's prediction on any given example back to specific influential training examples. Existing approaches do so by assign…
Towards Tracing Factual Knowledge in Language Models Back to the Training Data
Ekin Akyürek, Tolga Bolukbasi, Frederick Liu +4
Language models (LMs) have been shown to memorize a great deal of factual knowledge contained in their training data. But when an LM generates an assertion, it is often difficult t…
Retrieval-guided Counterfactual Generation for QA
Bhargavi Paranjape, Matthew Lamm, Ian Tenney
Deep NLP models have been shown to learn spurious correlations, leaving them brittle to input perturbations. Recent work has shown that counterfactual or contrastive data -- i.e. m…
The MultiBERTs: BERT Reproductions for Robustness Analysis
Thibault Sellam, Steve Yadlowsky, Jason Wei +9
Experiments with pre-trained models such as BERT are often based on a single checkpoint. While the conclusions drawn apply to the artifact tested in the experiment (i.e., the parti…
Do Language Embeddings Capture Scales?
Xikun Zhang, Deepak Ramachandran, Ian Tenney +2
Pretrained Language Models (LMs) have been shown to possess significant linguistic, common sense, and factual knowledge. One form of knowledge that has not been studied yet in this…
Measuring and Reducing Gendered Correlations in Pre-trained Models
Kellie Webster, Xuezhi Wang, Ian Tenney +6
Pre-trained models have revolutionized natural language understanding. However, researchers have found they can encode artifacts undesired in many applications, such as professions…
The Language Interpretability Tool: Extensible, Interactive Visualizations and Analysis for NLP Models
Ian Tenney, James Wexler, Jasmijn Bastings +8
We present the Language Interpretability Tool (LIT), an open-source platform for visualization and understanding of NLP models. We focus on core questions about model behavior: Why…
What Happens To BERT Embeddings During Fine-tuning?
Amil Merchant, Elahe Rahimtoroghi, Ellie Pavlick +1
While there has been much recent work studying how linguistic information is encoded in pre-trained sentence representations, comparatively little is understood about how these mod…
Asking without Telling: Exploring Latent Ontologies in Contextual Representations
Julian Michael, Jan A. Botha, Ian Tenney
The success of pretrained contextual encoders, such as ELMo and BERT, has brought a great deal of interest in what these models learn: do they, without explicit supervision, learn…
jiant: A Software Toolkit for Research on General-Purpose Text Understanding Models
Yada Pruksachatkun, Phil Yeres, Haokun Liu +5
We introduce jiant, an open source toolkit for conducting multitask and transfer learning experiments on English NLU tasks. jiant enables modular and configuration-driven experimen…
What do you learn from context? Probing for sentence structure in contextualized word representations
Ian Tenney, Patrick Xia, Berlin Chen +8
Contextualized representation models such as ELMo (Peters et al., 2018a) and BERT (Devlin et al., 2018) have recently achieved state-of-the-art results on a diverse array of downst…
BERT Rediscovers the Classical NLP Pipeline
Ian Tenney, Dipanjan Das, Ellie Pavlick
Pre-trained text encoders have rapidly advanced the state of the art on many NLP tasks. We focus on one such model, BERT, and aim to quantify where linguistic information is captur…
Probing What Different NLP Tasks Teach Machines about Function Word Comprehension
Najoung Kim, Roma Patel, Adam Poliak +9
We introduce a set of nine challenge tasks that test for the understanding of function words. These tasks are created by structurally mutating sentences from existing datasets to t…
Can You Tell Me How to Get Past Sesame Street? Sentence-Level Pretraining Beyond Language Modeling
Alex Wang, Jan Hula, Patrick Xia +13
Natural language understanding has recently seen a surge of progress with the use of sentence encoders like ELMo (Peters et al., 2018a) and BERT (Devlin et al., 2019) which are pre…
WikiAtomicEdits: A Multilingual Corpus of Wikipedia Edits for Modeling Language and Discourse
Manaal Faruqui, Ellie Pavlick, Ian Tenney +1
We release a corpus of 43 million atomic edits across 8 languages. These edits are mined from Wikipedia edit history and consist of instances in which a human editor has inserted a…
Delayed Ultrafast X-ray Auger Probing (DUXAP) of Nucleobase Ultraviolet Photoprotection
B. K. McFarland, J. P. Farrell, S. Miyabe +28
We present a new method for ultrafast spectroscopy of molecular photoexcited dynamics. The technique uses a pair of femtosecond pulses: a photoexcitation pulse initiating excited s…