29 citations · 107 across the 28 of their papers we have counts for
28 papers
Let the Batch Decide, On- or Off-Policy: Effective Sample Size for RL Post-Training
Rasool Fakoor, Murdock Aubry, Nicholas Stranges +1
Reinforcement learning (RL) is structurally harder than supervised learning because the policy changes the data distribution it learns from. The resulting fragility is especially v…
Understanding the Challenges in Iterative Generative Optimization with LLMs
Allen Nie, Xavier Daull, Zhiyi Kuang +10
Generative optimization uses large language models (LLMs) to iteratively improve artifacts (such as code, workflows or prompts) using execution feedback. It is a promising approach…
Offline Learning and Forgetting for Reasoning with Large Language Models
Tianwei Ni, Allen Nie, Sapana Chaudhary +3
Leveraging inference-time search in large language models has proven effective in further enhancing a trained model's capability to solve complex mathematical and reasoning problem…
From Demonstrations to Rewards: Alignment Without Explicit Human Preferences
Siliang Zeng, Yao Liu, Huzefa Rangwala +3
One of the challenges of aligning large models with human preferences lies in both the data requirements and the technical complexities of current approaches. Predominant methods,…
AlphaRouter: Quantum Circuit Routing with Reinforcement Learning and Tree Search
Wei Tang, Yiheng Duan, Yaroslav Kharkov +3
Quantum computers have the potential to outperform classical computers in important tasks such as optimization and number factoring. They are characterized by limited connectivity,…
Bridging the Training-Inference Gap in LLMs by Leveraging Self-Generated Tokens
Zhepeng Cen, Yao Liu, Siliang Zeng +4
Language models are often trained to maximize the likelihood of the next token given past tokens in the training dataset. However, during inference time, they are utilized differen…
AgentOccam: A Simple Yet Strong Baseline for LLM-Based Web Agents
Ke Yang, Yao Liu, Sapana Chaudhary +4
Autonomy via agents using large language models (LLMs) for personalized, standardized tasks boosts human efficiency. Automating web tasks (like booking hotels within a budget) is i…
EXTRACT: Efficient Policy Learning by Extracting Transferable Robot Skills from Offline Data
Jesse Zhang, Minho Heo, Zuxin Liu +4
Most reinforcement learning (RL) methods focus on learning optimal policies over low-level action spaces. While these methods can perform well in their training environments, they…
Learning the Target Network in Function Space
Kavosh Asadi, Yao Liu, Shoham Sabach +2
We focus on the task of learning the value function in the reinforcement learning (RL) setting. This task is often solved by updating a pair of online and target networks while ens…
TAIL: Task-specific Adapters for Imitation Learning with Large Pretrained Models
Zuxin Liu, Jesse Zhang, Kavosh Asadi +4
The full potential of large pretrained models remains largely untapped in control domains like robotics. This is mainly because of the scarcity of data and the computational challe…
Budgeting Counterfactual for Offline RL
Yao Liu, Pratik Chaudhari, Rasool Fakoor
The main challenge of offline reinforcement learning, where data is limited, arises from a sequence of counterfactual reasoning dilemmas within the realm of potential actions: What…
Resetting the Optimizer in Deep RL: An Empirical Study
Kavosh Asadi, Rasool Fakoor, Shoham Sabach
We focus on the task of approximating the optimal value function in deep reinforcement learning. This iterative process is comprised of solving a sequence of optimization problems…
TD Convergence: An Optimization Perspective
Kavosh Asadi, Shoham Sabach, Yao Liu +2
We study the convergence behavior of the celebrated temporal-difference (TD) learning algorithm. By looking at the algorithm through the lens of optimization, we first argue that T…
Time-Varying Propensity Score to Bridge the Gap between the Past and Present
Rasool Fakoor, Jonas Mueller, Zachary C. Lipton +2
Real-world deployment of machine learning models is challenging because data evolves over time. While no model can work when data evolves in an arbitrary fashion, if there is some…
Task-Agnostic Continual Reinforcement Learning: Gaining Insights and Overcoming Challenges
Massimo Caccia, Jonas Mueller, Taesup Kim +2
Continual learning (CL) enables the development of models and agents that learn from a sequence of tasks while addressing the limitations of standard deep learning approaches, such…
Faster Deep Reinforcement Learning with Slower Online Network
Kavosh Asadi, Rasool Fakoor, Omer Gottesman +3
Deep reinforcement learning algorithms often use two networks for value function optimization: an online network, and a target network that tracks the online network with some dela…
Flexible Model Aggregation for Quantile Regression
Rasool Fakoor, Taesup Kim, Jonas Mueller +2
Quantile regression is a fundamental problem in statistical learning motivated by a need to quantify uncertainty in predictions, or to model a diverse population without being over…
Continuous Doubly Constrained Batch Reinforcement Learning
Rasool Fakoor, Jonas Mueller, Kavosh Asadi +2
Reliant on too many experiments to learn good actions, current Reinforcement Learning (RL) algorithms have limited applicability in real-world settings, which can be too expensive…
DDPG++: Striving for Simplicity in Continuous-control Off-Policy Reinforcement Learning
Rasool Fakoor, Pratik Chaudhari, Alexander J. Smola
This paper prescribes a suite of techniques for off-policy Reinforcement Learning (RL) that simplify the training process and reduce the sample complexity. First, we show that simp…
Fast, Accurate, and Simple Models for Tabular Data via Augmented Distillation
Rasool Fakoor, Jonas Mueller, Nick Erickson +2
Automated machine learning (AutoML) can produce complex model ensembles by stacking, bagging, and boosting many individual models like trees, deep networks, and nearest neighbor es…
TraDE: Transformers for Density Estimation
Rasool Fakoor, Pratik Chaudhari, Jonas Mueller +1
We present TraDE, a self-attention-based architecture for auto-regressive density estimation with continuous and discrete valued data. Our model is trained using a penalized maximu…
Meta-Q-Learning
Rasool Fakoor, Pratik Chaudhari, Stefano Soatto +1
This paper introduces Meta-Q-Learning (MQL), a new off-policy algorithm for meta-Reinforcement Learning (meta-RL). MQL builds upon three simple ideas. First, we show that Q-learnin…
P3O: Policy-on Policy-off Policy Optimization
Rasool Fakoor, Pratik Chaudhari, Alexander J. Smola
On-policy reinforcement learning (RL) algorithms have high sample complexity while off-policy algorithms are difficult to tune. Merging the two holds the promise to develop efficie…
Differentiable Greedy Networks
Thomas Powers, Rasool Fakoor, Siamak Shakeri +4
Optimal selection of a subset of items from a given set is a hard problem that requires combinatorial optimization. In this paper, we propose a subset selection algorithm that is t…