18 citations · 75 across the 21 of their papers we have counts for
25 papers
High-Value Training Data from Noisy, Diverse Sources
Hedi Driss, Evan Frick, Anastasios N. Angelopoulos +4
Training state-of-the-art models increasingly requires learning from large, heterogeneous datasets whose examples vary in quality, domain relevance, and source of supervision. Data…
Corrupt Plans, Clean Traces: Evading Chain-of-Thought Monitoring with Plan Injection
Keertana Chidambaram, Andrew Ilyas, Vasilis Syrgkanis
Chain-of-thought (CoT) monitoring is a safety strategy where the reasoning of a large language model "actor" is inspected by a "monitor" (often another language model) for signs of…
Data Attribution via Sketched Metadifferentiation
Yuxi Chen, Hamza Golubovic, Han Tong +2
Data attribution seeks to quantify how individual training examples shape a model's predictions and underpins problems including data valuation, machine unlearning, and model inter…
Pick Your Poison: Learning to Select Poison Sets for Stronger LLM Backdoor Attacks
Aashiq Muhamed, Mona T. Diab, Virginia Smith +2
Backdoor poisoning attacks add poisoned examples to otherwise-clean finetuning data, pairing a trigger with a target behavior that the model learns to produce when the trigger appe…
Using AI Agents to Automate Black-Box Audits of Personalization Algorithms at Scale
Alessandro Morosini, Sarah H. Cen, Andrew Ilyas +3
Personalization algorithms determine what content users encounter on online platforms. Auditing these systems is difficult because independent auditors have only black-box access t…
Justified or Just Convincing? Error Verifiability as a Dimension of LLM Quality
Xiaoyuan Zhu, Kimberly Le Truong, Riccardo Fogliato +8
As LLMs are deployed in high-stakes settings, users must judge the correctness of individual responses, often relying on model-generated justifications such as reasoning chains or…
Large-Scale, Longitudinal Study of Large Language Models During the 2024 US Election Season
Sarah H. Cen, Andrew Ilyas, Hedi Driss +4
The 2024 US presidential election is the first major contest to occur in the US since the popularization of large language models (LLMs). Building on lessons from earlier shifts in…
Probably Approximately Correct Labels
Emmanuel J. Candès, Andrew Ilyas, Tijana Zrnic
Obtaining high-quality labeled datasets is often costly, requiring either human annotation or expensive experiments. In theory, powerful pre-trained AI models provide an opportunit…
DataMIL: Selecting Data for Robot Imitation Learning with Datamodels
Shivin Dass, Alaa Khaddaj, Logan Engstrom +3
Recently, the robotics community has amassed ever larger and more diverse datasets to train generalist policies. However, while these policies achieve strong mean performance acros…
AI Supply Chains: An Emerging Ecosystem of AI Actors, Products, and Services
Aspen Hopkins, Sarah H. Cen, Andrew Ilyas +3
The widespread adoption of AI in recent years has led to the emergence of AI supply chains: complex networks of AI actors contributing models, datasets, and more to the development…
MAGIC: Near-Optimal Data Attribution for Deep Learning
Andrew Ilyas, Logan Engstrom
The goal of predictive data attribution is to estimate how adding or removing a given set of training datapoints will affect model predictions. In convex settings, this goal is str…
Optimizing ML Training with Metagradient Descent
Logan Engstrom, Andrew Ilyas, Benjamin Chen +3
A major challenge in training large-scale machine learning models is configuring the training process to maximize model performance, i.e., finding the best training setup from a va…
Attribute-to-Delete: Machine Unlearning via Datamodel Matching
Kristian Georgiev, Roy Rinberg, Sung Min Park +4
Machine unlearning -- efficiently removing the effect of a small "forget set" of training data on a pre-trained machine learning model -- has recently attracted significant researc…
Data Debiasing with Datamodels (D3M): Improving Subgroup Robustness via Data Selection
Saachi Jain, Kimia Hamidieh, Kristian Georgiev +3
Machine learning models can fail on subgroups that are underrepresented during training. While techniques such as dataset balancing can improve performance on underperforming group…
Measuring Strategization in Recommendation: Users Adapt Their Behavior to Shape Future Content
Sarah H. Cen, Andrew Ilyas, Jennifer Allen +2
Most modern recommendation algorithms are data-driven: they generate personalized recommendations by observing users' past behaviors. A common assumption in recommendation is that…
Decomposing and Editing Predictions by Modeling Model Computation
Harshay Shah, Andrew Ilyas, Aleksander Madry
How does the internal computation of a machine learning model transform inputs into predictions? In this paper, we introduce a task called component modeling that aims to address t…
User Strategization and Trustworthy Algorithms
Sarah H. Cen, Andrew Ilyas, Aleksander Madry
Many human-facing algorithms -- including those that power recommender systems or hiring decision tools -- are trained on data provided by their users. The developers of these algo…
Rethinking Backdoor Attacks
Alaa Khaddaj, Guillaume Leclerc, Aleksandar Makelov +4
In a backdoor attack, an adversary inserts maliciously constructed backdoor examples into a training set to make the resulting model vulnerable to manipulation. Defending against s…
FFCV: Accelerating Training by Removing Data Bottlenecks
Guillaume Leclerc, Andrew Ilyas, Logan Engstrom +3
We present FFCV, a library for easy and fast machine learning model training. FFCV speeds up model training by eliminating (often subtle) data bottlenecks from the training process…
TRAK: Attributing Model Behavior at Scale
Sung Min Park, Kristian Georgiev, Andrew Ilyas +2
The goal of data attribution is to trace model predictions back to training data. Despite a long line of work towards this goal, existing approaches to data attribution tend to for…
Raising the Cost of Malicious AI-Powered Image Editing
Hadi Salman, Alaa Khaddaj, Guillaume Leclerc +2
We present an approach to mitigating the risks of malicious image editing posed by large diffusion models. The key idea is to immunize images so as to make them resistant to manipu…
ModelDiff: A Framework for Comparing Learning Algorithms
Harshay Shah, Sung Min Park, Andrew Ilyas +1
We study the problem of (learning) algorithm comparison, where the goal is to find differences between models trained with two different learning algorithms. We begin by formalizin…
When does Bias Transfer in Transfer Learning?
Hadi Salman, Saachi Jain, Andrew Ilyas +3
Using transfer learning to adapt a pre-trained "source model" to a downstream "target task" can dramatically increase performance with seemingly no downside. In this work, we demon…
What Makes A Good Fisherman? Linear Regression under Self-Selection Bias
Yeshwanth Cherapanamjeri, Constantinos Daskalakis, Andrew Ilyas +1
In the classical setting of self-selection, the goal is to learn models, simultaneously from observations where is the output of one of under…