27 citations · 83 across the 20 of their papers we have counts for
20 papers
Scaling Cross-Embodiment World Models for Dexterous Manipulation
Zihao He, Bo Ai, Tongzhou Mu +6
Cross-embodiment learning seeks to build generalist robots that learn from and operate across diverse morphologies, but differences in kinematics and action spaces hinder data shar…
Towards Embodiment Scaling Laws in Robot Locomotion
Bo Ai, Liu Dai, Nico Bohlinger +7
Cross-embodiment generalization underpins the vision of building generalist embodied agents for any robot, yet its enabling factors remain poorly understood. We investigate embodim…
Multi-Stage Manipulation with Demonstration-Augmented Reward, Policy, and World Model Learning
Adrià López Escoriza, Nicklas Hansen, Stone Tao +2
Long-horizon tasks in robotic manipulation present significant challenges in reinforcement learning (RL) due to the difficulty of designing dense reward functions and effectively e…
Responsive Noise-Relaying Diffusion Policy: Responsive and Efficient Visuomotor Control
Zhuoqun Chen, Xiu Yuan, Tongzhou Mu +1
Imitation learning is an efficient method for teaching robots a variety of tasks. Diffusion Policy, which uses a conditional denoising diffusion process to generate actions, has de…
When Should We Prefer State-to-Visual DAgger Over Visual Reinforcement Learning?
Tongzhou Mu, Zhaoyang Li, Stanisław Wiktor Strzelecki +4
Learning policies from high-dimensional visual inputs, such as pixels and point clouds, is crucial in various applications. Visual reinforcement learning is a promising approach th…
Policy Decorator: Model-Agnostic Online Refinement for Large Policy Model
Xiu Yuan, Tongzhou Mu, Stone Tao +3
Recent advancements in robot learning have used imitation learning with large models and extensive demonstrations to develop effective policies. However, these models are often lim…
ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AI
Stone Tao, Fanbo Xiang, Arth Shukla +20
Simulation has enabled unprecedented compute-scalable approaches to robot learning. However, many existing simulation frameworks typically support a narrow range of scenes/tasks an…
DrS: Learning Reusable Dense Rewards for Multi-Stage Tasks
Tongzhou Mu, Minghua Liu, Hao Su
The success of many RL techniques heavily relies on human-engineered dense rewards, which typically demand substantial domain expertise and extensive trial and error. In our work,…
AdaDemo: Data-Efficient Demonstration Expansion for Generalist Robotic Agent
Tongzhou Mu, Yijie Guo, Jie Xu +4
Encouraged by the remarkable achievements of language and vision foundation models, developing generalist robotic agents through imitation learning, using large demonstration datas…
Unleashing the Creative Mind: Language Model As Hierarchical Policy For Improved Exploration on Challenging Problem Solving
Zhan Ling, Yunhao Fang, Xuanlin Li +5
Large Language Models (LLMs) have achieved tremendous progress, yet they still often struggle with challenging reasoning problems. Current approaches address this challenge by samp…
Accelerated Doubly Stochastic Gradient Algorithm for Large-scale Empirical Risk Minimization
Zebang Shen, Hui Qian, Tongzhou Mu +1
Nowadays, algorithms with fast convergence, small memory footprints, and low per-iteration complexity are particularly favorable for artificial intelligence applications. In this p…
Boosting Reinforcement Learning and Planning with Demonstrations: A Survey
Tongzhou Mu, Hao Su
Although reinforcement learning has seen tremendous success recently, this kind of trial-and-error learning can be impractical or inefficient in complex environments. The use of de…
ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skills
Jiayuan Gu, Fanbo Xiang, Xuanlin Li +12
Generalizable manipulation skills, which can be composed to tackle long-horizon and complex daily chores, are one of the cornerstones of Embodied AI. However, existing benchmarks,…
On Pre-Training for Visuo-Motor Control: Revisiting a Learning-from-Scratch Baseline
Nicklas Hansen, Zhecheng Yuan, Yanjie Ze +5
In this paper, we examine the effectiveness of pre-training for visuo-motor control tasks. We revisit a simple Learning-from-Scratch (LfS) baseline that incorporates data augmentat…
Abstract-to-Executable Trajectory Translation for One-Shot Task Generalization
Stone Tao, Xiaochen Li, Tongzhou Mu +3
Training long-horizon robotic policies in complex physical environments is essential for many applications, such as robotic manipulation. However, learning a policy that can genera…
Close the Optical Sensing Domain Gap by Physics-Grounded Active Stereo Sensor Simulation
Xiaoshuai Zhang, Rui Chen, Ang Li +11
In this paper, we focus on the simulation of active stereovision depth sensors, which are popular in both academic and industry communities. Inspired by the underlying mechanism of…
Learning Two-Step Hybrid Policy for Graph-Based Interpretable Reinforcement Learning
Tongzhou Mu, Kaixiang Lin, Feiyang Niu +1
We present a two-step hybrid reinforcement learning (RL) policy that is designed to generate interpretable and robust hierarchical policies on the RL problem with graph-based input…
ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrations
Tongzhou Mu, Zhan Ling, Fanbo Xiang +6
Object manipulation from 3D visual inputs poses many challenges on building generalizable perception and policy models. However, 3D assets in existing benchmarks mostly lack the di…
Refactoring Policy for Compositional Generalizability using Self-Supervised Object Proposals
Tongzhou Mu, Jiayuan Gu, Zhiwei Jia +2
We study how to learn a policy with compositional generalizability. We propose a two-stage framework, which refactorizes a high-reward teacher policy into a generalizable student p…
State Alignment-based Imitation Learning
Fangchen Liu, Zhan Ling, Tongzhou Mu +1
Consider an imitation learning problem that the imitator and the expert have different dynamics models. Most of the current imitation learning methods fail because they focus on im…