112 citations · 304 across the 149 of their papers we have counts for
41 papers · 1 filter
FACT: A Multimodal Force-Action-Track Foundation Model for Human-to-Robot Dexterity
Zhi Cao, Yining Hong, Jiaolong Yang +4
Human demonstrations offer a scalable alternative to costly robot data collection for dexterous manipulation, but most human datasets lack force measurements. Dexterous manipulatio…
I3L: Iterative Interactive Imitation Learning for Whole-Body Mobile Manipulation
Wensi Ai, Christopher Luey, Karthik Dharmarajan +7
Policies trained offline from teleoperated demonstrations can degrade outside their training distribution. This gap is particularly acute in long-horizon whole-body mobile manipula…
The Human--VLA Communication Gap: What Humans Leave Implicit and What Policies Cannot Recover
Yiqing Xu, Jiajun Wu
A robot may know how to perform a task yet fail when a human asks for it naturally. We study this human--VLA communication gap by separating an \emph{information gap}, where task s…
One Demonstration, Many Objects: Generalizing Manipulation via Local Contact Geometry
Satvik Sharma, Samrat Sahoo, Huang Huang +4
Dexterous manipulation with multi-fingered robot hands promises human-level dexterity, but collecting large-scale dexterous robot hand data remains difficult. Learning from human d…
TrAct: Bridging Robot Control and Visual Prediction with Visual Tracks
Zhi Cao, Howard Ji, Kevin Zhang +4
Robot actions are inherently embodiment-specific and only weakly aligned with image-space visual changes, limiting their effectiveness as conditioning signals for robot world model…
CoStream: Composing Simple Behaviors for Generalizable Complex Manipulation
Haonan Chen, Yuxiang Ma, Stephen Tian +7
Long-horizon, contact-rich complex manipulation tasks, such as seating a GPU into a PCIe slot, demand both millimeter high precision and out-of-the-box generalization to new tasks.…
InSight: Self-Guided Skill Acquisition via Steerable VLAs
Maggie Wang, Lars Osterberg, Stephen Tian +5
Vision-language-action (VLA) models excel at robot manipulation via imitation learning, but adapting them to new tasks often requires additional human demonstrations, which can be…
DIRECT: When and Where Should You Allocate Test-Time Compute in Embodied Planners?
Jadelynn Dao, Milan Ganai, Yasmina Abukhadra +7
Vision-Language Models (VLMs) are increasingly deployed as high-level planners for embodied agents, with an emerging strategy of scaling test-time compute to improve capability. Ho…
StereoPolicy: Improving Robotic Manipulation Policies via Stereo Perception
Evans Han, Yunfan Jiang, Yingke Wang +6
Recent advances in robot imitation learning have produced powerful visuomotor policies that manipulate diverse objects from visual inputs. However, monocular observations lack dept…
World Model for Robot Learning: A Comprehensive Survey
Bohan Hou, Gen Li, Jindou Jia +15
World models, which are predictive representations of how environments evolve under actions, have become a central component of robot learning. They support policy learning, planni…
IMPASTO: Integrating Model-Based Planning with Learned Dynamics Models for Robotic Oil Painting Reproduction
Yingke Wang, Hao Li, Yifeng Zhu +6
Robotic reproduction of oil paintings using soft brushes and pigments requires force-sensitive control of deformable tools, prediction of brushstroke effects, and multi-step stroke…
CaP-X: A Framework for Benchmarking and Improving Coding Agents for Robot Manipulation
Letian Fu, Justin Yu, Karim El-Refai +13
"Code-as-Policy" considers how executable code can complement data-intensive Vision-Language-Action (VLA) methods, yet their effectiveness as autonomous controllers for embodied ma…
Dream2Flow: Bridging Video Generation and Open-World Manipulation with 3D Object Flow
Karthik Dharmarajan, Wenlong Huang, Jiajun Wu +2
Generative video modeling has emerged as a compelling tool to zero-shot reason about plausible physical interactions for open-world manipulation. Yet, it remains a challenge to tra…
Model-Based Policy Adaptation for Closed-Loop End-to-End Autonomous Driving
Haohong Lin, Yunzhi Zhang, Wenhao Ding +2
End-to-end (E2E) autonomous driving models have demonstrated strong performance in open-loop evaluations but often suffer from cascading errors and poor generalization in closed-lo…
10 Open Challenges Steering the Future of Vision-Language-Action Models
Soujanya Poria, Navonil Majumder, Chia-Yu Hung +7
Due to their ability of follow natural language instructions, vision-language-action (VLA) models are increasingly prevalent in the embodied AI arena, following the widespread succ…
TWIST2: Scalable, Portable, and Holistic Humanoid Data Collection System
Yanjie Ze, Siheng Zhao, Weizhuo Wang +6
Large-scale data has driven breakthroughs in robotics, from language models to vision-language-action models in bimanual manipulation. However, humanoid robotics lacks equally effe…
Retargeting Matters: General Motion Retargeting for Humanoid Motion Tracking
Joao Pedro Araujo, Yanjie Ze, Pei Xu +2
Humanoid motion tracking policies are central to building teleoperation pipelines and hierarchical controllers, yet they face a fundamental challenge: the embodiment gap between hu…
MoMaGen: Generating Demonstrations under Soft and Hard Constraints for Multi-Step Bimanual Mobile Manipulation
Chengshu Li, Mengdi Xu, Arpit Bahety +11
Imitation learning from large-scale, diverse human demonstrations has been shown to be effective for training robots, but collecting such data is costly and time-consuming. This ch…
Phys2Real: Fusing VLM Priors with Interactive Online Adaptation for Uncertainty-Aware Sim-to-Real Manipulation
Maggie Wang, Stephen Tian, Aiden Swann +3
Learning robotic manipulation policies directly in the real world can be expensive and time-consuming. While reinforcement learning (RL) policies trained in simulation present a sc…
DexSkin: High-Coverage Conformable Robotic Skin for Learning Contact-Rich Manipulation
Suzannah Wistreich, Baiyu Shi, Stephen Tian +5
Human skin provides a rich tactile sensing stream, localizing intentional and unintentional contact events over a large and contoured region. Replicating these tactile sensing capa…
VisualMimic: Visual Humanoid Loco-Manipulation via Motion Tracking and Generation
Shaofeng Yin, Yanjie Ze, Hong-Xing Yu +2
Humanoid loco-manipulation in unstructured environments demands tight integration of egocentric perception and whole-body control. However, existing approaches either depend on ext…
UAD: Unsupervised Affordance Distillation for Generalization in Robotic Manipulation
Yihe Tang, Wenlong Huang, Yingke Wang +5
Understanding fine-grained object affordances is imperative for robots to manipulate objects in unstructured environments given open-ended task instructions. However, existing meth…
Learning Compositional Behaviors from Demonstration and Language
Weiyu Liu, Neil Nie, Ruohan Zhang +2
We introduce Behavior from Language and Demonstration (BLADE), a framework for long-horizon robotic manipulation by integrating imitation learning and model-based planning. BLADE l…
TWIST: Teleoperated Whole-Body Imitation System
Yanjie Ze, Zixuan Chen, João Pedro Araújo +4
Teleoperating humanoid robots in a whole-body manner marks a fundamental step toward developing general-purpose robotic intelligence, with human motion providing an ideal interface…