30 citations · 78 across the 4 of their papers we have counts for
8 papers · 1 filter
Accuracy on the Line: On the Strong Correlation Between Out-of-Distribution and In-Distribution Generalization
John Miller, Rohan Taori, Aditi Raghunathan +6
For machine learning systems to be reliable, we must understand their performance in unseen, out-of-distribution environments. In this paper, we empirically show that out-of-distri…
Outside the Echo Chamber: Optimizing the Performative Risk
John Miller, Juan C. Perdomo, Tijana Zrnic
In performative prediction, predictions guide decision-making and hence can influence the distribution of future data. To date, work on performative prediction has focused on findi…
The Effect of Natural Distribution Shift on Question Answering Models
John Miller, Karl Krauth, Benjamin Recht +1
We build four new test sets for the Stanford Question Answering Dataset (SQuAD) and evaluate the ability of question-answering systems to generalize to new data. Our first test set…
Strategic Classification is Causal Modeling in Disguise
John Miller, Smitha Milli, Moritz Hardt
Consequential decision-making incentivizes individuals to strategically adapt their behavior to the specifics of the decision rule. While a long line of work has viewed strategic a…
Test-Time Training with Self-Supervision for Generalization under Distribution Shifts
Yu Sun, Xiaolong Wang, Zhuang Liu +3
In this paper, we propose Test-Time Training, a general approach for improving the performance of predictive models when training and test data come from different distributions. W…
Model Similarity Mitigates Test Set Overuse
Horia Mania, John Miller, Ludwig Schmidt +2
Excessive reuse of test data has become commonplace in today's machine learning workflows. Popular benchmarks, competitions, industrial scale tuning, among other applications, all…