52 citations · 194 across the 16 of their papers we have counts for
17 papers
DexTaG: Tactile-as-Guidance in Reinforcement Learning for Dexterous Manipulation
Han Yang, Yian Wang, Yunlong Song +2
Glove-based motion capture is emerging as a scalable approach to collecting dexterous-hand demonstration data. However, due to the kinematic gap between the human and robot hand, t…
Flow as the Cross-Domain Manipulation Interface
Mengda Xu, Zhenjia Xu, Yinghao Xu +4
We present Im2Flow2Act, a scalable learning framework that enables robots to acquire real-world manipulation skills without the need of real-world robot training data. The key idea…
DoughNet: A Visual Predictive Model for Topological Manipulation of Deformable Objects
Dominik Bauer, Zhenjia Xu, Shuran Song
Manipulation of elastoplastic objects like dough often involves topological changes such as splitting and merging. The ability to accurately predict these topological changes that…
Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots
Cheng Chi, Zhenjia Xu, Chuer Pan +5
We present Universal Manipulation Interface (UMI) -- a data collection and policy learning framework that allows direct skill transfer from in-the-wild human demonstrations to depl…
XSkill: Cross Embodiment Skill Discovery
Mengda Xu, Zhenjia Xu, Cheng Chi +2
Human demonstration videos are a widely available data source for robot learning and an intuitive user interface for expressing desired behavior. However, directly extracting reusa…
Robots That Ask For Help: Uncertainty Alignment for Large Language Model Planners
Allen Z. Ren, Anushri Dixit, Alexandra Bodrova +11
Large language models (LLMs) exhibit a wide range of promising capabilities -- from step-by-step planning to commonsense reasoning -- that may provide utility for robots, but remai…
Towards Generalist Robots: A Promising Paradigm via Generative Simulation
Zhou Xian, Theophile Gervet, Zhenjia Xu +3
This document serves as a position paper that outlines the authors' vision for a potential pathway towards generalist robots. The purpose of this document is to share the excitemen…
FluidLab: A Differentiable Environment for Benchmarking Complex Fluid Manipulation
Zhou Xian, Bo Zhu, Zhenjia Xu +4
Humans manipulate various kinds of fluids in their everyday life: creating latte art, scooping floating objects from water, rolling an ice cream cone, etc. Using robots to augment…
Diffusion Policy: Visuomotor Policy Learning via Action Diffusion
Cheng Chi, Zhenjia Xu, Siyuan Feng +5
This paper introduces Diffusion Policy, a new way of generating robot behavior by representing a robot's visuomotor policy as a conditional denoising diffusion process. We benchmar…
RoboNinja: Learning an Adaptive Cutting Policy for Multi-Material Objects
Zhenjia Xu, Zhou Xian, Xingyu Lin +4
We introduce RoboNinja, a learning-based cutting system for multi-material objects (i.e., soft objects with rigid cores such as avocados or mangos). In contrast to prior works usin…
BusyBot: Learning to Interact, Reason, and Plan in a BusyBoard Environment
Zeyi Liu, Zhenjia Xu, Shuran Song
We introduce BusyBoard, a toy-inspired robot learning environment that leverages a diverse set of articulated objects and inter-object functional relations to provide rich visual f…
DextAIRity: Deformable Manipulation Can be a Breeze
Zhenjia Xu, Cheng Chi, Benjamin Burchfiel +3
This paper introduces DextAIRity, an approach to manipulate deformable objects using active airflow. In contrast to conventional contact-based quasi-static manipulations, DextAIRit…
UMPNet: Universal Manipulation Policy Network for Articulated Objects
Zhenjia Xu, Zhanpeng He, Shuran Song
We introduce the Universal Manipulation Policy Network (UMPNet) -- a single image-based policy network that infers closed-loop action sequences for manipulating arbitrary articulat…
Learning 3D Dynamic Scene Representations for Robot Manipulation
Zhenjia Xu, Zhanpeng He, Jiajun Wu +1
3D scene representation for robot manipulation should capture three key object properties: permanency -- objects that become occluded over time continue to exist; amodal completene…
AdaGrasp: Learning an Adaptive Gripper-Aware Grasping Policy
Zhenjia Xu, Beichun Qi, Shubham Agrawal +1
This paper aims to improve robots' versatility and adaptability by allowing them to use a large variety of end-effector tools and quickly adapt to new tools. We propose AdaGrasp, a…
DensePhysNet: Learning Dense Physical Object Representations via Multi-step Dynamic Interactions
Zhenjia Xu, Jiajun Wu, Andy Zeng +2
We study the problem of learning physical object representations for robot manipulation. Understanding object physics is critical for successful object manipulation, but also chall…
Unsupervised Discovery of Parts, Structure, and Dynamics
Zhenjia Xu, Zhijian Liu, Chen Sun +4
Humans easily recognize object parts and their hierarchical structure by watching how they move; they can then predict how each part moves in the future. In this paper, we propose…