11 citations · 47 across the 26 of their papers we have counts for
13 papers · 1 filter
Learning Robust Reasoning through Guided Adversarial Self-Play
Shuozhe Li, Vaishnav Tadiparthi, Kwonjoon Lee +6
Reinforcement learning from verifiable rewards (RLVR) produces strong reasoning models, yet they can fail catastrophically when the conditioning context is fallible (e.g., corrupte…
ExPO: Unlocking Hard Reasoning with Self-Explanation-Guided Reinforcement Learning
Ruiyang Zhou, Shuozhe Li, Amy Zhang +1
Self-improvement via RL often fails on complex reasoning tasks because GRPO-style post-training methods rely on the model's initial ability to generate positive samples. Without gu…
Linear Representation Transferability Hypothesis: Leveraging Small Models to Steer Large Models
Femi Bello, Anubrata Das, Fanzhi Zeng +2
It has been hypothesized that neural networks with similar architectures trained on similar data learn shared representations relevant to the learning task. We build on this idea b…
A Common Pitfall of Margin-based Language Model Alignment: Gradient Entanglement
Hui Yuan, Yifan Zeng, Yue Wu +3
Reinforcement Learning from Human Feedback (RLHF) has become the predominant approach for language model (LM) alignment. At its core, RLHF uses a margin-based loss for preference o…
Accounting for AI and Users Shaping One Another: The Role of Mathematical Models
Sarah Dean, Evan Dong, Meena Jagadeesan +1
As AI systems enter into a growing number of societal domains, these systems increasingly shape and are shaped by user preferences, opinions, and behaviors. However, the design of…
Off-Policy Risk Assessment in Markov Decision Processes
Audrey Huang, Liu Leqi, Zachary Chase Lipton +1
Addressing such diverse ends as safety alignment with human preferences, and the efficiency of learning, a growing line of reinforcement learning research focuses on risk functiona…
Modeling Attrition in Recommender Systems with Departing Bandits
Omer Ben-Porat, Lee Cohen, Liu Leqi +2
Traditionally, when recommender systems are formalized as multi-armed bandits, the policy of the recommender system influences the rewards accrued, but not the length of interactio…
Action-Sufficient State Representation Learning for Control with Structural Constraints
Biwei Huang, Chaochao Lu, Liu Leqi +4
Perceived signals in real-world scenarios are usually high-dimensional and noisy, and finding and using their representation that contains essential and sufficient information requ…
Off-Policy Risk Assessment in Contextual Bandits
Audrey Huang, Liu Leqi, Zachary C. Lipton +1
Even when unable to run experiments, practitioners can evaluate prospective policies, using previously logged data. However, while the bandits literature has adopted a diverse set…
On the Convergence and Optimality of Policy Gradient for Markov Coherent Risk
Audrey Huang, Liu Leqi, Zachary C. Lipton +1
In order to model risk aversion in reinforcement learning, an emerging line of research adapts familiar algorithms to optimize coherent risk functionals, a class that includes cond…
Rebounding Bandits for Modeling Satiation Effects
Liu Leqi, Fatma Kilinc-Karzan, Zachary C. Lipton +1
Psychological research shows that enjoyment of many goods is subject to satiation, with short-term satisfaction declining after repeated exposures to the same item. Nevertheless, p…
Game Design for Eliciting Distinguishable Behavior
Fan Yang, Liu Leqi, Yifan Wu +4
The ability to inferring latent psychological traits from human behavior is key to developing personalized human-interacting machine learning systems. Approaches to infer such trai…
Automated Dependence Plots
David I. Inouye, Liu Leqi, Joon Sik Kim +2
In practical applications of machine learning, it is necessary to look beyond standard metrics such as test accuracy in order to validate various qualitative properties of a model.…