99 citations · 183 across the 27 of their papers we have counts for
17 papers · 1 filter
MaD-RL: Matching Distributions for Calibrating LLMs with Reinforcement Learning
Sourabh Kulkarni, Ksheeraj Sai Vepuri, Basar Demir +8
Reinforcement learning (RL) is widely used in language-model post-training to maximize rewards assigned to individual model outputs, such as scores from binary verifiers or reward…
Inverse RL Helps Align AI by Imitating Humans
Michał Wiliński, Liu Leqi, Chirag Nagpal
Language model alignment aims to make model behavior reflect desirable properties such as helpfulness, safety, and instruction following. Existing methods draw supervision from exp…
Discretizing Reward Models
Vijay Viswanathan, Shiqi Wang, Devamanyu Hazarika +4
Despite their widespread use, the role of reward models in shaping reinforcement learning is poorly understood. Reward models offer a tempting promise: they automatically estimate…
Utility-inspired Reward Transformations Improve Reinforcement Learning Training of Language Models
Roberto-Rafael Maura-Rivero, Chirag Nagpal, Roma Patel +1
Current methods that train large language models (LLMs) with reinforcement learning feedback, often resort to averaging outputs of multiple rewards functions during training. This…
InfAlign: Inference-aware language model alignment
Ananth Balashankar, Ziteng Sun, Jonathan Berant +9
Language model alignment is a critical step in training modern generative language models. Alignment targets to improve win rate of a sample from the aligned model against the base…
Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoning
Amrith Setlur, Chirag Nagpal, Adam Fisch +6
A promising approach for improving reasoning in large language models is to use process reward models (PRMs). PRMs provide feedback at each step of a multi-step reasoning trace, po…
Robust Preference Optimization through Reward Model Distillation
Adam Fisch, Jacob Eisenstein, Vicky Zayats +5
Language model (LM) post-training (or alignment) involves maximizing a reward function that is derived from preference annotations. Direct Preference Optimization (DPO) is a popula…
Theoretical guarantees on the best-of-n alignment policy
Ahmad Beirami, Alekh Agarwal, Jonathan Berant +4
A simple and effective method for the inference-time alignment and scaling test-time compute of generative models is best-of- sampling, where samples are drawn from a refere…
Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate Reward Hacking
Jacob Eisenstein, Chirag Nagpal, Alekh Agarwal +9
Reward models play a key role in aligning language model applications towards human preferences. However, this setup creates an incentive for the language model to exploit errors i…
Participatory Personalization in Classification
Hailey Joren, Chirag Nagpal, Katherine Heller +1
Machine learning models are often personalized with information that is protected, sensitive, self-reported, or costly to acquire. These models use information about people but do…
auton-survival: an Open-Source Package for Regression, Counterfactual Estimation, Evaluation and Phenotyping with Censored Time-to-Event Data
Chirag Nagpal, Willa Potosnak, Artur Dubrawski
Applications of machine learning in healthcare often require working with time-to-event prediction tasks including prognostication of an adverse event, re-hospitalization or death.…
Counterfactual Phenotyping with Censored Time-to-Events
Chirag Nagpal, Mononito Goswami, Keith Dufendach +1
Estimation of treatment efficacy of real-world clinical interventions involves working with continuous outcomes such as time-to-death, re-hospitalization, or a composite event that…
Deep Cox Mixtures for Survival Regression
Chirag Nagpal, Steve Yadlowsky, Negar Rostamzadeh +1
Survival analysis is a challenging variation of regression modeling because of the presence of censoring, where the outcome measurement is only partially known, due to, for example…
Deep Survival Machines: Fully Parametric Survival Regression and Representation Learning for Censored Data with Competing Risks
Chirag Nagpal, Xinyu Rachel Li, Artur Dubrawski
We describe a new approach to estimating relative risks in time-to-event prediction problems with censored data in a fully parametric manner. Our approach does not require making s…
Nonlinear Semi-Parametric Models for Survival Analysis
Chirag Nagpal, Rohan Sangave, Amit Chahar +3
Semi-parametric survival analysis methods like the Cox Proportional Hazards (CPH) regression (Cox, 1972) are a popular approach for survival analysis. These methods involve fitting…
Interpretable Subgroup Discovery in Treatment Effect Estimation with Application to Opioid Prescribing Guidelines
Chirag Nagpal, Dennis Wei, Bhanukiran Vinzamuri +4
The dearth of prescribing guidelines for physicians is one key driver of the current opioid epidemic in the United States. In this work, we analyze medical and pharmaceutical claim…
Preserving Intermediate Objectives: One Simple Trick to Improve Learning for Hierarchical Models
Abhilasha Ravichander, Shruti Rijhwani, Rajat Kulshreshtha +3
Hierarchical models are utilized in a wide variety of problems which are characterized by task hierarchies, where predictions on smaller subtasks are useful for trying to predict a…