2.3k citations · 4.9k across the 24 of their papers we have counts for
16 papers · 1 filter
Proximity-Informed Calibration for Deep Neural Networks
Miao Xiong, Ailin Deng, Pang Wei Koh +4
Confidence calibration is central to providing accurate and interpretable uncertainty estimates, especially under safety-critical scenarios. However, we find that existing calibrat…
On the Trade-off of Intra-/Inter-class Diversity for Supervised Pre-training
Jieyu Zhang, Bohan Wang, Zhengyu Hu +2
Pre-training datasets are critical for building state-of-the-art machine learning models, motivating rigorous study on their impact on downstream tasks. In this work, we study the…
Out-of-Domain Robustness via Targeted Augmentations
Irena Gao, Shiori Sagawa, Pang Wei Koh +2
Models trained on one set of domains often suffer performance drops on unseen domains, e.g., when wildlife monitoring models are deployed in new camera locations. In this work, we…
Improving Domain Generalization with Domain Relations
Huaxiu Yao, Xinyu Yang, Xinyi Pan +3
Distribution shift presents a significant challenge in machine learning, where models often underperform during the test stage when faced with a different distribution than the one…
Impossibility Theorems for Feature Attribution
Blair Bilodeau, Natasha Jaques, Pang Wei Koh +1
Despite a sea of interpretability methods that can produce plausible explanations, the field has also empirically seen many failure cases of such methods. In light of these results…
Wild-Time: A Benchmark of in-the-Wild Distribution Shift over Time
Huaxiu Yao, Caroline Choi, Bochuan Cao +3
Distribution shift occurs when the test distribution differs from the training distribution, and it can considerably degrade performance of machine learning models deployed in the…
Extending the WILDS Benchmark for Unsupervised Adaptation
Shiori Sagawa, Pang Wei Koh, Tony Lee +17
Machine learning systems deployed in the wild are often trained on a source distribution but deployed on a different target distribution. Unlabeled data can be a powerful point of…
On the Opportunities and Risks of Foundation Models
Rishi Bommasani, Drew A. Hudson, Ehsan Adeli +111
AI is undergoing a paradigm shift with the rise of models (e.g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks.…
Just Train Twice: Improving Group Robustness without Training Group Information
Evan Zheran Liu, Behzad Haghgoo, Annie S. Chen +5
Standard training via empirical risk minimization (ERM) can produce models that achieve high accuracy on average but low accuracy on certain groups, especially in the presence of s…
Accuracy on the Line: On the Strong Correlation Between Out-of-Distribution and In-Distribution Generalization
John Miller, Rohan Taori, Aditi Raghunathan +6
For machine learning systems to be reliable, we must understand their performance in unseen, out-of-distribution environments. In this paper, we empirically show that out-of-distri…
Concept Bottleneck Models
Pang Wei Koh, Thao Nguyen, Yew Siang Tang +4
We seek to learn models that we can interact with using high-level concepts: if the model did not think there was a bone spur in the x-ray, would it still predict severe arthritis?…
Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization
Shiori Sagawa, Pang Wei Koh, Tatsunori B. Hashimoto +1
Overparameterized neural networks can be highly accurate on average on an i.i.d. test set yet consistently fail on atypical groups of the data (e.g., by learning spurious correlati…
Temporal FiLM: Capturing Long-Range Sequence Dependencies with Feature-Wise Modulations
Sawyer Birnbaum, Volodymyr Kuleshov, Zayd Enam +2
Learning representations that accurately capture long-range dependencies in sequential inputs -- including text, audio, and genomic data -- is a key problem in deep learning. Feed-…
On the Accuracy of Influence Functions for Measuring Group Effects
Pang Wei Koh, Kai-Siang Ang, Hubert H. K. Teo +1
Influence functions estimate the effect of removing a training point on a model without the need to retrain. They are based on a first-order Taylor approximation that is guaranteed…
Inferring Multidimensional Rates of Aging from Cross-Sectional Data
Emma Pierson, Pang Wei Koh, Tatsunori Hashimoto +4
Modeling how individuals evolve over time is a fundamental problem in the natural and social sciences. However, existing datasets are often cross-sectional with each individual obs…
Certified Defenses for Data Poisoning Attacks
Jacob Steinhardt, Pang Wei Koh, Percy Liang
Machine learning systems trained on user-provided data are susceptible to data poisoning attacks, whereby malicious users inject false training data with the aim of corrupting the…