13 papers
ForkCast: Predictive Recovery for Vision-Language-Action Policies via Counterfactual Futures
Chenru Wen, Huajian Zeng, Haokai Ding +5
Vision-language-action (VLA) policies perform manipulation tasks using camera images and language instructions, but object slip or visual occlusion can disrupt execution. Recovery…
Exo2EgoHOI: Hand-Object-Interaction Aware Exocentric-to-Egocentric Video Generation
Hongjia Zhai, Xiyu Zhang, Haoran Zhang +5
Egocentric videos of human manipulation provide valuable visual experience for embodied intelligence, yet collecting such data at scale is costly. Exocentric-to-egocentric video ge…
The Robot Data Factory
Sami Haddadin, Ivan Laptev, Ian Reid +18
Physical AI requires more than increasingly large robot datasets: intelligent robots acquire knowledge through continuous interaction with the physical world. We argue that the def…
SURE-Map: Self-Correcting Streaming Geometric Foundation Models
Mingkai Liu, Hao Zhao, Xingxing Zuo
Streaming geometric foundation models are emerging as a compelling alternative to SLAM systems. Yet this streaming nature introduces a fundamental issue: each prediction is made fr…
World-Action Models for Robot Learning and Control: A Survey
Zuxing Lu, Hongjia Zhai, Guanzhi Wang +13
Robots operating in open environments act under partial observability, physical constraints, and dynamic task contexts. Beyond mapping observations and language instructions to act…
MAC-I: Learned Metrics-Aware Covariance for Robust Visual-Inertial Fusion in Initialization and Calibration
Xiang Fei, Yuheng Qiu, Can Xu +5
Visual-Inertial (VI) fusion is fundamental to accurate and robust state estimation, where camera and IMU measurements are combined according to their respective uncertainties. Exis…
LooperMuscle: Fast and Stable Learning of Humanoid Whole-Body Tracking via Structured Mixture-of-Experts
Boyi Liu, Qijin Li, Tianqi Yu +2
FastSAC-style methods significantly reduce humanoid motion training time but often suffer from notable performance degradation compared with PPO in whole-body tracking tasks. We ta…
M3imic: Learning a Versatile Whole-Body Controller for Multimodal Motion Mimicking
Zuxing Lu, Ziang Zheng, Yao Lyu +7
Building a general-purpose whole-body controller is essential for enabling diverse motion capabilities in humanoid robots across a wide range of downstream tasks, including locomot…
VAG: Dual-Stream Video-Action Generation for Embodied Data Synthesis
Xiaolei Lang, Yang Wang, Yukun Zhou +10
Recent advances in robot foundation models trained on large-scale human teleoperation data have enabled robots to perform increasingly complex real-world tasks. However, scaling th…
EgoFlow: Gradient-Guided Flow Matching for Egocentric 6DoF Object Motion Generation
Abhishek Saroha, Huajian Zeng, Xingxing Zuo +2
Understanding and predicting object motion from egocentric video is fundamental to embodied perception and interaction. However, generating physically consistent 6DoF trajectories…
SC-Diff: 3D Shape Completion with Latent Diffusion Models
Simon Schaefer, Juan D. Galvis, Xingxing Zuo +1
We present a novel 3D shape completion framework that unifies multimodal conditioning, leveraging both 2D images and 3D partial scans through a latent diffusion model. Shapes are r…
NeRF-VO: Real-Time Sparse Visual Odometry with Neural Radiance Fields
Jens Naumann, Binbin Xu, Stefan Leutenegger +1
We introduce a novel monocular visual odometry (VO) system, NeRF-VO, that integrates learning-based sparse visual odometry for low-latency camera tracking and a neural radiance sce…
Dynamic LiDAR Re-simulation using Compositional Neural Fields
Hanfeng Wu, Xingxing Zuo, Stefan Leutenegger +3
We introduce DyNFL, a novel neural field-based approach for high-fidelity re-simulation of LiDAR scans in dynamic driving scenes. DyNFL processes LiDAR measurements from dynamic en…