430 citations · 512 across the 19 of their papers we have counts for
3 papers · 1 filter
Pretraining Data Mixtures Enable Narrow Model Selection Capabilities in Transformer Models
Steve Yadlowsky, Lyric Doshi, Nilesh Tripuraneni
Transformer models, notably large language models (LLMs), have the remarkable ability to perform in-context learning (ICL) -- to perform new tasks when prompted with unseen input-o…
Choosing a Proxy Metric from Past Experiments
Nilesh Tripuraneni, Lee Richardson, Alexander D'Amour +2
In many randomized experiments, the treatment effect of the long-term metric (i.e. the primary outcome of interest) is often difficult or infeasible to measure. Such long-term metr…
Diagnosing Model Performance Under Distribution Shift
Tiffany Tianhui Cai, Hongseok Namkoong, Steve Yadlowsky
Prediction models can perform poorly when deployed to target distributions different from the training distribution. To understand these operational failure modes, we develop a met…