36 citations · 165 across the 31 of their papers we have counts for
21 papers · 1 filter
Off-Context GRPO: Learning to Reason on Hard Problems using Privileged Information
Priyank Agrawal, Ankur Samanta, Shervin Ghasemlou +5
Reinforcement learning with verifiable rewards (RLVR) improves reasoning in large language models. Yet, typical RLVR approaches fail on difficult problems: when a model cannot gene…
Displacement-Resistant Extensions of DPO with Nonconvex -Divergences
Idan Pipano, Shoham Sabach, Kavosh Asadi +1
DPO and related algorithms align language models by directly optimizing the RLHF objective: find a policy that maximizes the Bradley-Terry reward while staying close to a reference…
Learning to Reason Efficiently with Discounted Reinforcement Learning
Alex Ayoub, Kavosh Asadi, Dale Schuurmans +2
Large reasoning models (LRMs) often consume excessive tokens, inflating computational cost and latency. More broadly, in goal reaching sequential decision problems we often want to…
C2-DPO: Constrained Controlled Direct Preference Optimization
Kavosh Asadi, Julien Han, Idan Pipano +5
Direct preference optimization (\texttt{DPO}) has emerged as a promising approach for solving the alignment problem in AI. In this paper, we make two counter-intuitive observations…
Adjoint sharding for very long context training of state space models
Xingzi Xu, Amir Tavanaei, Kavosh Asadi +1
Despite very fast progress, efficiently training large language models (LLMs) in very long contexts remains challenging. Existing methods fall back to training LLMs with short cont…
Learning the Target Network in Function Space
Kavosh Asadi, Yao Liu, Shoham Sabach +2
We focus on the task of learning the value function in the reinforcement learning (RL) setting. This task is often solved by updating a pair of online and target networks while ens…
TAIL: Task-specific Adapters for Imitation Learning with Large Pretrained Models
Zuxin Liu, Jesse Zhang, Kavosh Asadi +4
The full potential of large pretrained models remains largely untapped in control domains like robotics. This is mainly because of the scarcity of data and the computational challe…
Resetting the Optimizer in Deep RL: An Empirical Study
Kavosh Asadi, Rasool Fakoor, Shoham Sabach
We focus on the task of approximating the optimal value function in deep reinforcement learning. This iterative process is comprised of solving a sequence of optimization problems…
TD Convergence: An Optimization Perspective
Kavosh Asadi, Shoham Sabach, Yao Liu +2
We study the convergence behavior of the celebrated temporal-difference (TD) learning algorithm. By looking at the algorithm through the lens of optimization, we first argue that T…
Faster Deep Reinforcement Learning with Slower Online Network
Kavosh Asadi, Rasool Fakoor, Omer Gottesman +3
Deep reinforcement learning algorithms often use two networks for value function optimization: an online network, and a target network that tracks the online network with some dela…
Coarse-Grained Smoothness for RL in Metric Spaces
Omer Gottesman, Kavosh Asadi, Cameron Allen +3
Principled decision-making in continuous state--action spaces is impossible without some assumptions. A common approach is to assume Lipschitz continuity of the Q-function. We show…
Convergence of a Human-in-the-Loop Policy-Gradient Algorithm With Eligibility Trace Under Reward, Policy, and Advantage Feedback
Ishaan Shah, David Halpern, Kavosh Asadi +1
Fluid human-agent communication is essential for the future of human-in-the-loop reinforcement learning. An agent must respond appropriately to feedback from its human trainer even…
Continuous Doubly Constrained Batch Reinforcement Learning
Rasool Fakoor, Jonas Mueller, Kavosh Asadi +2
Reliant on too many experiments to learn good actions, current Reinforcement Learning (RL) algorithms have limited applicability in real-world settings, which can be too expensive…
Learning State Abstractions for Transfer in Continuous Control
Kavosh Asadi, David Abel, Michael L. Littman
Can simple algorithms with a good representation solve challenging reinforcement learning problems? In this work, we answer this question in the affirmative, where we take "simple…
Deep Radial-Basis Value Functions for Continuous Control
Kavosh Asadi, Neev Parikh, Ronald E. Parr +2
A core operation in reinforcement learning (RL) is finding an action that is optimal with respect to a learned value function. This operation is often challenging when the learned…
Lipschitz Lifelong Reinforcement Learning
Erwan Lecarpentier, David Abel, Kavosh Asadi +3
We consider the problem of knowledge transfer when an agent is facing a series of Reinforcement Learning (RL) tasks. We introduce a novel metric between Markov Decision Processes (…
Combating the Compounding-Error Problem with a Multi-step Model
Kavosh Asadi, Dipendra Misra, Seungchan Kim +1
Model-based reinforcement learning is an appealing framework for creating agents that learn, plan, and act in sequential environments. Model-based algorithms typically involve lear…
Mitigating Planner Overfitting in Model-Based Reinforcement Learning
Dilip Arumugam, David Abel, Kavosh Asadi +5
An agent with an inaccurate model of its environment faces a difficult choice: it can ignore the errors in its model and act in the real world in whatever way it determines is opti…
Towards a Simple Approach to Multi-step Model-based Reinforcement Learning
Kavosh Asadi, Evan Cater, Dipendra Misra +1
When environmental interaction is expensive, model-based reinforcement learning offers a solution by planning ahead and avoiding costly mistakes. Model-based agents typically learn…
Equivalence Between Wasserstein and Value-Aware Loss for Model-based Reinforcement Learning
Kavosh Asadi, Evan Cater, Dipendra Misra +1
Learning a generative model is a key component of model-based reinforcement learning. Though learning a good model in the tabular setting is a simple task, learning a useful model…
Lipschitz Continuity in Model-based Reinforcement Learning
Kavosh Asadi, Dipendra Misra, Michael L. Littman
We examine the impact of learning Lipschitz continuous models in the context of model-based reinforcement learning. We provide a novel bound on multi-step prediction error of Lipsc…