2.3k citations · 11.9k across the 167 of their papers we have counts for
20 papers · 1 filter
Learning Autocomplete Systems as a Communication Game
Mina Lee, Tatsunori B. Hashimoto, Percy Liang
We study textual autocomplete---the task of predicting a full sentence from a partial sentence---as a human-machine communication game. Specifically, we consider three competing go…
Feature Noise Induces Loss Discrepancy Across Groups
Fereshte Khani, Percy Liang
The performance of standard learning procedures has been observed to differ widely across groups. Recent studies usually attribute this loss discrepancy to an information deficienc…
Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization
Shiori Sagawa, Pang Wei Koh, Tatsunori B. Hashimoto +1
Overparameterized neural networks can be highly accurate on average on an i.i.d. test set yet consistently fail on atypical groups of the data (e.g., by learning spurious correlati…
Shaping Visual Representations with Language for Few-shot Classification
Jesse Mu, Percy Liang, Noah Goodman
By describing the features and abstractions of our world, language is a crucial tool for human learning and a promising source of supervision for machine learning models. We use la…
Designing and Interpreting Probes with Control Tasks
John Hewitt, Percy Liang
Probes, supervised models trained to predict properties (like parts-of-speech) from representations (like ELMo), have achieved high accuracy on a range of linguistic tasks. But doe…
Distributionally Robust Language Modeling
Yonatan Oren, Shiori Sagawa, Tatsunori B. Hashimoto +1
Language models are generally trained on data spanning a wide range of topics (e.g., news, reviews, fiction), but they might be applied to an a priori unknown target distribution (…
Certified Robustness to Adversarial Word Substitutions
Robin Jia, Aditi Raghunathan, Kerem Göksel +1
State-of-the-art NLP models can often be fooled by adversaries that apply seemingly innocuous label-preserving transformations (e.g., paraphrasing) to input text. The number of pos…
Verified Uncertainty Calibration
Ananya Kumar, Percy Liang, Tengyu Ma
Applications such as weather forecasting and personalized medicine demand models that output calibrated probability estimates---those representative of the true likelihood of a pre…
SPoC: Search-based Pseudocode to Code
Sumith Kulal, Panupong Pasupat, Kartik Chandra +4
We consider the task of mapping pseudocode to long programs that are functionally correct. Given test cases as a mechanism to validate programs, we search over the space of possibl…
Maximum Weighted Loss Discrepancy
Fereshte Khani, Aditi Raghunathan, Percy Liang
Though machine learning algorithms excel at minimizing the average loss over a population, this might lead to large discrepancies between the losses across groups within the popula…
Selection via Proxy: Efficient Data Selection for Deep Learning
Cody Coleman, Christopher Yeh, Stephen Mussmann +5
Data selection methods, such as active learning and core-set selection, are useful tools for machine learning on large datasets. However, they can be prohibitively expensive to app…
A Tight Analysis of Greedy Yields Subexponential Time Approximation for Uniform Decision Tree
Ray Li, Percy Liang, Stephen Mussmann
Decision Tree is a classic formulation of active learning: given hypotheses with nonnegative weights summing to 1 and a set of tests that each partition the hypotheses, output…
Adversarial Training Can Hurt Generalization
Aditi Raghunathan, Sang Michael Xie, Fanny Yang +2
While adversarial training can improve robust accuracy (against an adversary), it sometimes hurts standard accuracy (when there is no adversary). Previous work has studied this tra…
Unlabeled Data Improves Adversarial Robustness
Yair Carmon, Aditi Raghunathan, Ludwig Schmidt +2
We demonstrate, theoretically and empirically, that adversarial robustness can significantly benefit from semisupervised learning. Theoretically, we revisit the simple Gaussian mod…
On the Accuracy of Influence Functions for Measuring Group Effects
Pang Wei Koh, Kai-Siang Ang, Hubert H. K. Teo +1
Influence functions estimate the effect of removing a training point on a model without the need to retrain. They are based on a first-order Taylor approximation that is guaranteed…
Strategies for Pre-training Graph Neural Networks
Weihua Hu, Bowen Liu, Joseph Gomes +4
Many applications of machine learning require a model to make accurate pre-dictions on test examples that are distributionally different from training ones, while task-specific lab…
Pun Generation with Surprise
He He, Nanyun Peng, Percy Liang
We tackle the problem of generating a pun sentence given a pair of homophones (e.g., "died" and "dyed"). Supervised text generation is inappropriate due to the lack of a large corp…
Unifying Human and Statistical Evaluation for Natural Language Generation
Tatsunori B. Hashimoto, Hugh Zhang, Percy Liang
How can we measure whether a natural language generation system produces both high quality and diverse outputs? Human evaluation captures quality but not diversity, as it does not…
Defending against Whitebox Adversarial Attacks via Randomized Discretization
Yuchen Zhang, Percy Liang
Adversarial perturbations dramatically decrease the accuracy of state-of-the-art image classifiers. In this paper, we propose and analyze a simple and computationally efficient def…
Ambitious Data Science Can Be Painless
Hatef Monajemi, Riccardo Murri, Eric Jonas +3
Modern data science research can involve massive computational experimentation; an ambitious PhD in computational fields may do experiments consuming several million CPU hours. Tra…