182 citations · 517 across the 17 of their papers we have counts for
12 papers · 1 filter
Prompts Generalize with Low Data: Non-vacuous Generalization Bounds for Optimizing Prompts with More Informative Priors
David Madras, Joshua Safyan, Qiuyi +1
Many prompt engineering techniques have been successful in practice, even when optimizing over a large prompt space with with a small amount of task-specific data. Recent work has…
QuEst: Enhancing Estimates of Quantile-Based Distributional Measures Using Model Predictions
Zhun Deng, Thomas P Zollo, Benjamin Eyre +3
As machine learning models grow increasingly competent, their predictions can supplement scarce or expensive data in various important domains. In support of this paradigm, algorit…
Regression for the Mean: Auto-Evaluation and Inference with Few Labels through Post-hoc Regression
Benjamin Eyre, David Madras
The availability of machine learning systems that can effectively perform arbitrary tasks has led to synthetic labels from these systems being used in applications of statistical i…
Out of the Ordinary: Spectrally Adapting Regression for Covariate Shift
Benjamin Eyre, Elliot Creager, David Madras +2
Designing deep neural network classifiers that perform robustly on distributions differing from the available training data is an active area of machine learning research. However,…
Identifying and Benchmarking Natural Out-of-Context Prediction Problems
David Madras, Richard Zemel
Deep learning systems frequently fail at out-of-context (OOC) prediction, the problem of making reliable predictions on uncommon or unusual inputs or subgroups of the training dist…
Fairness and Robustness in Invariant Learning: A Case Study in Toxicity Classification
Robert Adragna, Elliot Creager, David Madras +1
Robustness is of central importance in machine learning and has given rise to the fields of domain generalization and invariant learning, which are concerned with improving perform…
Amortized Causal Discovery: Learning to Infer Causal Graphs from Time-Series Data
Sindy Löwe, David Madras, Richard Zemel +1
On time-series data, most causal discovery methods fit a new model whenever they encounter samples from a new underlying causal graph. However, these samples often share relevant i…
Detecting Underspecification with Local Ensembles
David Madras, James Atwood, Alex D'Amour
We present local ensembles, a method for detecting underspecification -- when many possible predictors are consistent with the training data and model class -- at test time in a pr…
Causal Modeling for Fairness in Dynamical Systems
Elliot Creager, David Madras, Toniann Pitassi +1
In many application areas---lending, education, and online recommenders, for example---fairness and equity concerns emerge when a machine learning system interacts with a dynamical…
Flexibly Fair Representation Learning by Disentanglement
Elliot Creager, David Madras, Jörn-Henrik Jacobsen +4
We consider the problem of learning representations that achieve group and subgroup fairness with respect to multiple sensitive attributes. Taking inspiration from the disentangled…
Fairness Through Causal Awareness: Learning Latent-Variable Models for Biased Data
David Madras, Elliot Creager, Toniann Pitassi +1
How do we learn from biased data? Historical datasets often reflect historical prejudices; sensitive or protected attributes may affect the observed treatments and outcomes. Classi…
Learning Adversarially Fair and Transferable Representations
David Madras, Elliot Creager, Toniann Pitassi +1
In this paper, we advocate for representation learning as the key to mitigating unfair prediction outcomes downstream. Motivated by a scenario where learned representations are use…