Publications (16)
Learning Phonotactics from Linguistic Informants
Canaan Breiss, Alexis Ross, Amani Maina-Kilaas +2
We propose an interactive approach to language learning that utilizes linguistic acceptability judgments from an informant (a competent language user) to learn a grammar. Given a g…
Modeling Student Learning with 3.8 Million Program Traces
Alexis Ross, Megha Srivastava, Jeremiah Blanchard +1
As programmers write code, they often edit and retry multiple times, creating rich "interaction traces" that reveal how they approach coding tasks and provide clues about their lev…
ARIES: A Corpus of Scientific Paper Edits Made in Response to Peer Reviews
Mike D'Arcy, Alexis Ross, Erin Bransom +4
We introduce the task of automatically revising scientific papers based on peer feedback and release ARIES, a dataset of review comments and their corresponding paper edits. The da…
Inverse Scaling: When Bigger Isn't Better
Ian R. McKenzie, Alexander Lyzhov, Michael Pieler +24
Work on scaling laws has found that large language models (LMs) show predictable improvements to overall loss with increased scale (model size, training data, and compute). Here, w…
Does Self-Rationalization Improve Robustness to Spurious Correlations?
Alexis Ross, Matthew E. Peters, Ana MarasoviÄ
Rationalization is fundamental to human reasoning and learning. NLP models trained to produce rationales along with predictions, called self-rationalization models, have been inves…
CREST: A Joint Framework for Rationalization and Counterfactual Text Generation
Marcos Treviso, Alexis Ross, Nuno M. Guerreiro +1
Selective rationales and counterfactual examples have emerged as two effective, complementary classes of interpretability methods for analyzing and training NLP models. However, pr…
Learning to Make MISTAKEs: Modeling Incorrect Student Thinking And Key Errors
Alexis Ross, Jacob Andreas
Research on reasoning in language models (LMs) predominantly focuses on improving the correctness of their outputs. But some important applications require modeling reasoning patte…
Explaining NLP Models via Minimal Contrastive Editing (MiCE)
Alexis Ross, Ana MarasoviÄ, Matthew E. Peters
Humans have been shown to give contrastive explanations, which explain why an observed event happened rather than some other counterfactual event (the contrast case). Despite the i…
Competency Problems: On Finding and Removing Artifacts in Language Data
Matt Gardner, William Merrill, Jesse Dodge +4
Much recent work in NLP has documented dataset artifacts, bias, and spurious correlations between input features and output labels. However, how to tell which features have "spurio…
Toward In-Context Teaching: Adapting Examples to Students' Misconceptions
Alexis Ross, Jacob Andreas
When a teacher provides examples for a student to study, these examples must be informative, enabling a student to progress from their current state toward a target concept or skil…
Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks
Zhaofeng Wu, Linlu Qiu, Alexis Ross +6
The impressive performance of recent language models across a wide range of tasks suggests that they possess a degree of abstract reasoning skills. Are these skills general and tra…
Learning Models for Actionable Recourse
Alexis Ross, Himabindu Lakkaraju, Osbert Bastani
As machine learning models are increasingly deployed in high-stakes domains such as legal and financial decision-making, there has been growing interest in post-hoc methods for gen…
Tailor: Generating and Perturbing Text with Semantic Controls
Alexis Ross, Tongshuang Wu, Hao Peng +2
Controlled text perturbation is useful for evaluating and improving model generalizability. However, current techniques rely on training a model for every target perturbation, whic…
Completion Collaboration: Scaling Collaborative Effort with Agents
Shannon Zejiang Shen, Valerie Chen, Ken Gu +11
Current evaluations of agents remain centered around one-shot task completion, failing to account for the inherently iterative and collaborative nature of many real-world problems,…
Language Modeling with Editable External Knowledge
Belinda Z. Li, Emmy Liu, Alexis Ross +3
When the world changes, so does the text that humans write about it. How do we build language models that can be easily updated to reflect these changes? One popular approach is re…
Probing What Different NLP Tasks Teach Machines about Function Word Comprehension
Najoung Kim, Roma Patel, Adam Poliak +9
We introduce a set of nine challenge tasks that test for the understanding of function words. These tasks are created by structurally mutating sentences from existing datasets to t…