Nika Haghtalab

Inference Auctions

Keegan Harris, Siddharth Prasad, Asher Trockman +2

When inference demand exceeds available compute capacity, model providers must decide which requests should be served first. Users have different tolerances for delay from an LLM A…

Learn Your Own Thoughts: Abstract Token Curriculum

Khashayar Gatmiry, Avrajit Ghosh, Parsa Mirtaheri +4

Large Language Models (LLMs) have achieved remarkable reasoning capabilities by utilizing chain-of-thought (CoT) as a scratchpad for intermediate stages of thinking. However, CoT t…

Distortion of AI Alignment Revisited: RLHF is a Decent Utilitarian Aligner

Kazusato Oko, Annie Ulichney, Nika Haghtalab +1

While Reinforcement Learning from Human Feedback (RLHF) is the standard paradigm for aligning large language models with human preferences, its effectiveness in pluralistic setting…

Post-Training at the Edge of Detectability: A Game-Theoretic Approach to Fine-Tuning

Keegan Harris, Brian W. Lee, Ian Waudby-Smith +3

Reinforcement learning (RL) fine-tuning is widely used in language model training to improve performance on a target task while limiting drift from a reference policy. A standard w…

Provably Optimal Learning Algorithms for Assistance Games

Nivasini Ananthakrishnan, Mark Bedaywi, Michael I. Jordan +2

This paper studies an online variant of the assistance games framework, where an informed agent and an uninformed agent repeatedly interact over $T$ timesteps to optimize a common…

Blackwell Approachability and Gradient Equilibrium are Equivalent

Brian W. Lee, Nika Haghtalab, Michael I. Jordan +1

Gradient equilibrium (GEQ) is a recently introduced online optimization framework that generalizes first-order stationarity from offline optimization and abstracts problems like on…

Strategic Feature Selection

Jivat Neet Kaur, Pratik Patil, Divya Shanmugam +6

When algorithmic predictors inform resource allocation in high-stakes domains such as healthcare, these predictors must account for strategic manipulation of input features. The ty…

DiPOD: Diffusion Policy Optimization without Drifting Apart

Haozhe Jiang, Haiwen Feng, Pieter Abbeel +3

RL post-training has become increasingly pivotal for improving diffusion policies, but existing diffusion policy-gradient methods are often unstable and cannot achieve reliable pol…

Subliminal Effects in Your Data: A General Mechanism via Log-Linearity

Ishaq Aden-Ali, Noah Golowich, Allen Liu +3

Training modern large language models (LLMs) has become a veritable smorgasbord of algorithms and datasets designed to elicit particular behaviors, making it critical to develop te…

Diffusion Language Models are Provably Optimal Parallel Samplers

Haozhe Jiang, Nika Haghtalab, Lijie Chen

Diffusion language models (DLMs) have emerged as a promising alternative to autoregressive models for faster inference via parallel token generation. We provide a rigorous foundati…

Sample-Adaptivity Tradeoff in On-Demand Sampling

Nika Haghtalab, Omar Montasser, Mingda Qiao

We study the tradeoff between sample complexity and round complexity in on-demand sampling, where the learning algorithm adaptively samples from $k$ distributions over a limited nu…

Panprediction: Optimal Predictions for Any Downstream Task and Loss

Sivaraman Balakrishnan, Nika Haghtalab, Daniel Hsu +2

Supervised learning is classically formulated as training a model to minimize a fixed loss function over a fixed distribution, or task. However, an emerging paradigm instead views…

On Surjectivity of Neural Networks: Can you elicit any behavior from your model?

Haozhe Jiang, Nika Haghtalab

Given a trained neural network, can any specified output be generated by some input? Equivalently, does the network correspond to a function that is surjective? In generative model…

Distortion of AI Alignment: Does Preference Optimization Optimize for Preferences?

Paul Gölz, Nika Haghtalab, Kunhe Yang

After pre-training, large language models are aligned with human preferences based on pairwise comparisons. State-of-the-art alignment methods (such as PPO-based RLHF and DPO) are…

Learning With Multi-Group Guarantees For Clusterable Subpopulations

Jessica Dai, Nika Haghtalab, Eric Zhao

A canonical desideratum for prediction problems is that performance guarantees should hold not just on average over the population, but also for meaningful subpopulations within th…

Algorithmic Content Selection and the Impact of User Disengagement

Emilio Calvano, Nika Haghtalab, Ellen Vitercik +1

Digital services face a fundamental trade-off in content selection: they must balance the immediate revenue gained from high-reward content against the long-term benefits of mainta…

Truthfulness of Calibration Measures

Nika Haghtalab, Mingda Qiao, Kunhe Yang +1

We initiate the study of the truthfulness of calibration measures in sequential prediction. A calibration measure is said to be truthful if the forecaster (approximately) minimizes…