5 citations · 29 across the 29 of their papers we have counts for
29 papers
Learning to Act While Waiting: RL Finetuning of Generalist Robot Policies Under Inference Latency
Brian Zhu, Momen Khalil, E Harrison +17
While reinforcement learning (RL) allows generalist robot policies to continually improve during deployment, the large model size of modern generalist policies, such as VLAs, poses…
Why Does Action Chunking Improve Behavioral Cloning Performance in Robotic Control?
Filippo Lazzati, Kyle Stachowicz, William Chen +3
Action chunking---predicting and executing multiple actions instead of a single action---has proven to be a critical component for learning effective robotic control policies. Howe…
Adapting Generalist Robot Policies with Semantic Reinforcement Learning
Jagdeep Singh Bhatia, Andrew Wagenmaker, William Chen +1
Generalist robot policies learn a diverse repertoire of behaviors from large-scale pretraining. In principle, this makes them excellent priors for downstream adaptation via reinfor…
Learning Process Rewards via Success Visitation Matching for Efficient RL
Raymond Tsao, Andrew Wagenmaker, Sergey Levine
In many modern applications of reinforcement learning (RL), the natural reward for a task of interest is inherently sparse: a reward of 0 is given everywhere except when the task i…
Improving Robotic Generalist Policies via Flow Reversal Steering
Andy Tang, William Chen, Andrew Wagenmaker +2
Generalist policies can learn a wide range of skills from diverse robot datasets. In order to solve or improve on challenging new tasks, we need a way to infer and invoke the appro…
RoboReward: General-Purpose Vision-Language Reward Models for Robotics
Tony Lee, Andrew Wagenmaker, Karl Pertsch +3
A well-designed reward is critical for effective reinforcement learning-based policy improvement. In real-world robotics, obtaining such rewards typically requires either labor-int…
Posterior Behavioral Cloning: Pretraining BC Policies for Efficient RL Finetuning
Andrew Wagenmaker, Perry Dong, Raymond Tsao +2
Standard practice across domains from robotics to language is to first pretrain a policy on a large-scale demonstration dataset, and then finetune this policy, typically with reinf…
Robust Finetuning of Vision-Language-Action Robot Policies via Parameter Merging
Yajat Yadav, Zhiyuan Zhou, Andrew Wagenmaker +2
Generalist robot policies, trained on large and diverse datasets, have demonstrated the ability to generalize across a wide spectrum of behaviors, enabling a single policy to act i…
Behavioral Exploration: Learning to Explore via In-Context Adaptation
Andrew Wagenmaker, Zhiyuan Zhou, Sergey Levine
Developing autonomous agents that quickly explore an environment and adapt their behavior online is a canonical challenge in robotics and machine learning. While humans are able to…
Steering Your Diffusion Policy with Latent Space Reinforcement Learning
Andrew Wagenmaker, Mitsuhiko Nakamoto, Yunchu Zhang +5
Robotic control policies learned from human demonstrations have achieved impressive results in many real-world applications. However, in scenarios where initial performance is not…
Active learning of neural population dynamics using two-photon holographic optogenetics
Andrew Wagenmaker, Lu Mi, Marton Rozsa +5
Recent advances in techniques for monitoring and perturbing neural populations have greatly enhanced our ability to study circuits in the brain. In particular, two-photon holograph…
Overcoming the Sim-to-Real Gap: Leveraging Simulation to Learn to Explore for Real-World RL
Andrew Wagenmaker, Kevin Huang, Liyiming Ke +3
In order to mitigate the sample complexity of real-world reinforcement learning, common practice is to first train a policy in a simulator where samples are cheap, and then deploy…
Corruption-Robust Linear Bandits: Minimax Optimality and Gap-Dependent Misspecification
Haolin Liu, Artin Tajdini, Andrew Wagenmaker +1
In linear bandits, how can a learner effectively learn when facing corrupted rewards? While significant work has explored this question, a holistic understanding across different a…
Sample Complexity Reduction via Policy Difference Estimation in Tabular Reinforcement Learning
Adhyyan Narang, Andrew Wagenmaker, Lillian Ratliff +1
In this paper, we study the non-asymptotic sample complexity for the pure exploration problem in contextual bandits and tabular reinforcement learning (RL): identifying an epsilon-…
Humor in AI: Massive Scale Crowd-Sourced Preferences and Benchmarks for Cartoon Captioning
Jifan Zhang, Lalit Jain, Yang Guo +9
We present a novel multimodal preference dataset for creative tasks, consisting of over 250 million human ratings on more than 2.2 million captions, collected through crowdsourcing…
ASID: Active Exploration for System Identification in Robotic Manipulation
Marius Memmel, Andrew Wagenmaker, Chuning Zhu +3
Model-free control strategies such as reinforcement learning have shown the ability to learn control strategies without requiring an accurate model or simulator of the world. While…
Fair Active Learning in Low-Data Regimes
Romain Camilleri, Andrew Wagenmaker, Jamie Morgenstern +2
In critical machine learning applications, ensuring fairness is essential to avoid perpetuating social inequities. In this work, we address the challenges of reducing bias and impr…
Optimal Exploration for Model-Based RL in Nonlinear Systems
Andrew Wagenmaker, Guanya Shi, Kevin Jamieson
Learning to control unknown nonlinear dynamical systems is a fundamental problem in reinforcement learning and control theory. A commonly applied approach is to first explore the e…
Instance-Optimality in Interactive Decision Making: Toward a Non-Asymptotic Theory
Andrew Wagenmaker, Dylan J. Foster
We consider the development of adaptive, instance-dependent algorithms for interactive decision making (bandits, reinforcement learning, and beyond) that, rather than only performi…
Leveraging Offline Data in Online Reinforcement Learning
Andrew Wagenmaker, Aldo Pacchiano
Two central paradigms have emerged in the reinforcement learning (RL) community: online RL and offline RL. In the online RL setting, the agent has no prior knowledge of the environ…
Instance-Dependent Near-Optimal Policy Identification in Linear MDPs via Online Experiment Design
Andrew Wagenmaker, Kevin Jamieson
While much progress has been made in understanding the minimax sample complexity of reinforcement learning (RL) -- the complexity of learning on the "worst-case" instance -- such m…
Active Learning with Safety Constraints
Romain Camilleri, Andrew Wagenmaker, Jamie Morgenstern +2
Active learning methods have shown great promise in reducing the number of samples necessary for learning. As automated learning systems are adopted into real-time, real-world deci…
Reward-Free RL is No Harder Than Reward-Aware RL in Linear Markov Decision Processes
Andrew Wagenmaker, Yifang Chen, Max Simchowitz +2
Reward-free reinforcement learning (RL) considers the setting where the agent does not have access to a reward function during exploration, but must propose a near-optimal policy f…
First-Order Regret in Reinforcement Learning with Linear Function Approximation: A Robust Estimation Approach
Andrew Wagenmaker, Yifang Chen, Max Simchowitz +2
Obtaining first-order regret bounds -- regret bounds scaling not as the worst-case but with some measure of the performance of the optimal policy on a given instance -- is a core q…