232 citations · 300 across the 7 of their papers we have counts for
6 papers · 1 filter
Practical Benefits of Feature Feedback Under Distribution Shift
Anurag Katakkar, Clay H. Yoo, Weiqin Wang +2
In attempts to develop sample-efficient and interpretable algorithms, researcher have explored myriad mechanisms for collecting and exploiting feature feedback (or rationales) auxi…
On the Efficacy of Adversarial Data Collection for Question Answering: Results from a Large-Scale Randomized Study
Divyansh Kaushik, Douwe Kiela, Zachary C. Lipton +1
In adversarial data collection (ADC), a human workforce interacts with a model in real time, attempting to produce examples that elicit incorrect predictions. Researchers hope that…
Dynabench: Rethinking Benchmarking in NLP
Douwe Kiela, Max Bartolo, Yixin Nie +16
We introduce Dynabench, an open-source platform for dynamic dataset creation and model benchmarking. Dynabench runs in a web browser and supports human-and-model-in-the-loop datase…
Explaining The Efficacy of Counterfactually Augmented Data
Divyansh Kaushik, Amrith Setlur, Eduard Hovy +1
In attempts to produce ML models less reliant on spurious patterns in NLP datasets, researchers have recently proposed curating counterfactually augmented data (CAD) via a human-in…
Learning the Difference that Makes a Difference with Counterfactually-Augmented Data
Divyansh Kaushik, Eduard Hovy, Zachary C. Lipton
Despite alarm over the reliance of machine learning systems on so-called spurious patterns, the term lacks coherent meaning in standard statistical frameworks. However, the languag…
How Much Reading Does Reading Comprehension Require? A Critical Investigation of Popular Benchmarks
Divyansh Kaushik, Zachary C. Lipton
Many recent papers address reading comprehension, where examples consist of (question, passage, answer) tuples. Presumably, a model must combine information from both questions and…