9 citations · 22 across the 3 of their papers we have counts for
7 papers
Same Pre-training Loss, Better Downstream: Implicit Bias Matters for Language Models
Hong Liu, Sang Michael Xie, Zhiyuan Li +1
Language modeling on large-scale datasets leads to impressive performance gains on various downstream language tasks. The validation pre-training loss (or perplexity in autoregress…
No True State-of-the-Art? OOD Detection Methods are Inconsistent across Datasets
Fahim Tajwar, Ananya Kumar, Sang Michael Xie +1
Out-of-distribution detection is an important component of reliable ML systems. Prior literature has proposed various methods (e.g., MSP (Hendrycks & Gimpel, 2017), ODIN (Liang et…
In-N-Out: Pre-Training and Self-Training using Auxiliary Information for Out-of-Distribution Robustness
Sang Michael Xie, Ananya Kumar, Robbie Jones +3
Consider a prediction setting with few in-distribution labeled examples and many unlabeled examples both in- and out-of-distribution (OOD). The goal is to learn a model which perfo…
Understanding and Mitigating the Tradeoff Between Robustness and Accuracy
Aditi Raghunathan, Sang Michael Xie, Fanny Yang +2
Adversarial training augments the training set with perturbations to improve the robust error (over worst-case perturbations), but it often leads to an increase in the standard err…
Adversarial Training Can Hurt Generalization
Aditi Raghunathan, Sang Michael Xie, Fanny Yang +2
While adversarial training can improve robust accuracy (against an adversary), it sometimes hurts standard accuracy (when there is no adversary). Previous work has studied this tra…
Reparameterizable Subset Sampling via Continuous Relaxations
Sang Michael Xie, Stefano Ermon
Many machine learning tasks require sampling a subset of items from a collection based on a parameterized distribution. The Gumbel-softmax trick can be used to sample a single item…