2.3k citations · 11.9k across the 167 of their papers we have counts for
22 papers · 1 filter
Removing Spurious Features can Hurt Accuracy and Affect Groups Disproportionately
Fereshte Khani, Percy Liang
The presence of spurious features interferes with the goal of obtaining robust models that perform well across many groups within the population. A natural remedy is to remove spur…
In-N-Out: Pre-Training and Self-Training using Auxiliary Information for Out-of-Distribution Robustness
Sang Michael Xie, Ananya Kumar, Robbie Jones +3
Consider a prediction setting with few in-distribution labeled examples and many unlabeled examples both in- and out-of-distribution (OOD). The goal is to learn a model which perfo…
Enabling certification of verification-agnostic networks via memory-efficient semidefinite programming
Sumanth Dathathri, Krishnamurthy Dvijotham, Alexey Kurakin +8
Convex relaxations have emerged as a promising approach for verifying desirable properties of neural networks like robustness to adversarial perturbations. Widely used Linear Progr…
Beyond I.I.D.: Three Levels of Generalization for Question Answering on Knowledge Bases
Yu Gu, Sue Kase, Michelle Vanni +4
Existing studies on question answering on knowledge bases (KBQA) mainly operate with the standard i.i.d assumption, i.e., training distribution over questions is the same as the te…
RNNs can generate bounded hierarchical languages with optimal memory
John Hewitt, Michael Hahn, Surya Ganguli +2
Recurrent neural networks empirically generate natural language with high syntactic fidelity. However, their success is not well-understood theoretically. We provide theoretical in…
The EOS Decision and Length Extrapolation
Benjamin Newman, John Hewitt, Percy Liang +1
Extrapolation to unseen sequence lengths is a challenge for neural generative models of language. In this work, we characterize the effect on length extrapolation of a modeling dec…
Learning Adaptive Language Interfaces through Decomposition
Siddharth Karamcheti, Dorsa Sadigh, Percy Liang
Our goal is to create an interactive natural language interface that efficiently and reliably learns from users to complete tasks in simulated robotics settings. We introduce a neu…
On the Importance of Adaptive Data Collection for Extremely Imbalanced Pairwise Tasks
Stephen Mussmann, Robin Jia, Percy Liang
Many pairwise classification tasks, such as paraphrase detection and open-domain question answering, naturally have extreme label imbalance (e.g., of examples are negativ…
Selective Classification Can Magnify Disparities Across Groups
Erik Jones, Shiori Sagawa, Pang Wei Koh +2
Selective classification, in which models can abstain on uncertain predictions, is a natural approach to improving accuracy in settings where errors are costly but abstentions are…
Robustness to Spurious Correlations via Human Annotations
Megha Srivastava, Tatsunori Hashimoto, Percy Liang
The reliability of machine learning systems critically assumes that the associations between features and labels remain similar between training and test distributions. However, un…
Decoupling Exploration and Exploitation for Meta-Reinforcement Learning without Sacrifices
Evan Zheran Liu, Aditi Raghunathan, Percy Liang +1
The goal of meta-reinforcement learning (meta-RL) is to build agents that can quickly learn new tasks by leveraging prior experience on related tasks. Learning a new task often req…
Learning Abstract Models for Strategic Exploration and Fast Reward Transfer
Evan Zheran Liu, Ramtin Keramati, Sudarshan Seshadri +4
Model-based reinforcement learning (RL) is appealing because (i) it enables planning and thus more strategic exploration, and (ii) by decoupling dynamics from rewards, it enables f…
Concept Bottleneck Models
Pang Wei Koh, Thao Nguyen, Yew Siang Tang +4
We seek to learn models that we can interact with using high-level concepts: if the model did not think there was a bone spur in the x-ray, would it still predict severe arthritis?…
Graph-based, Self-Supervised Program Repair from Diagnostic Feedback
Michihiro Yasunaga, Percy Liang
We consider the problem of learning to repair programs from diagnostic feedback (e.g., compiler error messages). Program repair is challenging for two reasons: First, it requires r…
Selective Question Answering under Domain Shift
Amita Kamath, Robin Jia, Percy Liang
To avoid giving wrong answers, question answering (QA) models need to know when to abstain from answering. Moreover, users often ask questions that diverge from the model's trainin…
Composed Fine-Tuning: Freezing Pre-Trained Denoising Autoencoders for Improved Generalization
Sang Michael Xie, Tengyu Ma, Percy Liang
We focus on prediction problems with structured outputs that are subject to output validity constraints, e.g. pseudocode-to-code translation where the code must compile. While labe…
ExpBERT: Representation Engineering with Natural Language Explanations
Shikhar Murty, Pang Wei Koh, Percy Liang
Suppose we want to specify the inductive bias that married couples typically go on honeymoons for the task of extracting pairs of spouses from text. In this paper, we allow model d…
Robust Encodings: A Framework for Combating Adversarial Typos
Erik Jones, Robin Jia, Aditi Raghunathan +1
Despite excellent performance on many tasks, NLP systems are easily fooled by small adversarial perturbations of inputs. Existing procedures to defend against such perturbations ar…
Enabling Language Models to Fill in the Blanks
Chris Donahue, Mina Lee, Percy Liang
We present a simple approach for text infilling, the task of predicting missing spans of text at any position in a document. While infilling could enable rich functionality especia…
An Investigation of Why Overparameterization Exacerbates Spurious Correlations
Shiori Sagawa, Aditi Raghunathan, Pang Wei Koh +1
We study why overparameterization -- increasing model size well beyond the point of zero training error -- can hurt test error on minority groups despite improving average test err…
Understanding Self-Training for Gradual Domain Adaptation
Ananya Kumar, Tengyu Ma, Percy Liang
Machine learning systems must adapt to data distributions that evolve over time, in applications ranging from sensor networks and self-driving car perception modules to brain-machi…
Understanding and Mitigating the Tradeoff Between Robustness and Accuracy
Aditi Raghunathan, Sang Michael Xie, Fanny Yang +2
Adversarial training augments the training set with perturbations to improve the robust error (over worst-case perturbations), but it often leads to an increase in the standard err…