11 papers
GIFT: Guided Intermediate Feature Training via Action-Oriented Structural Supervision for Robotic Manipulation
Yupeng Zheng, Xiang Li, Songen Gu +11
Vision-language pre-training and predictive world modeling provide robot policies with rich semantic and dynamic visual features, but their native action and visual-prediction obje…
Latent Action as Intention Enables Efficient Future Imagination for World Action Models
Xiang Li, Yupeng Zheng, Songen Gu +11
World action models (WAMs) improve robot control by modeling how observations evolve, but generating future observations at test time incurs substantial latency. Fast-WAM removes t…
PokeVLA: Empowering Pocket-Sized Vision-Language-Action Model with Comprehensive World Knowledge Guidance
Yupeng Zheng, Xiang Li, Songen Gu +12
Recent advances in Vision-Language-Action (VLA) models have opened new avenues for robot manipulation, yet existing methods exhibit limited efficiency and a lack of high-level know…
VistaBot: View-Robust Robot Manipulation via Spatiotemporal-Aware View Synthesis
Songen Gu, Yuhang Zheng, Weize Li +6
Recently, end-to-end robotic manipulation models have gained significant attention for their generalizability and scalability. However, they often suffer from limited robustness to…
Latent-WAM: Latent World Action Modeling for End-to-End Autonomous Driving
Linbo Wang, Yupeng Zheng, Qiang Chen +14
We introduce Latent-WAM, an efficient end-to-end autonomous driving framework that achieves strong trajectory planning through spatially-aware and dynamics-informed latent world re…
OmniVTA: Visuo-Tactile World Modeling for Contact-Rich Robotic Manipulation
Yuhang Zheng, Songen Gu, Yupeng Zheng +11
Contact-rich manipulation tasks, such as wiping and assembly, require accurate perception of contact forces, friction changes, and state transitions that cannot be reliably inferre…
Enhancing Indoor Occupancy Prediction via Sparse Query-Based Multi-Level Consistent Knowledge Distillation
Xiang Li, Yupeng Zheng, Pengfei Li +3
Occupancy prediction provides critical geometric and semantic understanding for robotics but faces efficiency-accuracy trade-offs. Current dense methods suffer computational waste…
World In Your Hands: A Large-Scale and Open-Source Ecosystem for Learning Human-Centric Manipulation in the Wild
Yupeng Zheng, Jichao Peng, Weize Li +22
We introduce World In Your Hands (WIYH), a large-scale open-source ecosystem comprising over 1,000 hours of human manipulation data collected in-the-wild with millimeter-scale moti…
Semi-Supervised Vision-Centric 3D Occupancy World Model for Autonomous Driving
Xiang Li, Pengfei Li, Yupeng Zheng +3
Understanding world dynamics is crucial for planning in autonomous driving. Recent methods attempt to achieve this by learning a 3D occupancy world model that forecasts future surr…
HiPrompt: Tuning-free Higher-Resolution Generation with Hierarchical MLLM Prompts
Xinyu Liu, Yingqing He, Lanqing Guo +10
The potential for higher-resolution image generation using pretrained diffusion models is immense, yet these models often struggle with issues of object repetition and structural a…
MonoOcc: Digging into Monocular Semantic Occupancy Prediction
Yupeng Zheng, Xiang Li, Pengfei Li +6
Monocular Semantic Occupancy Prediction aims to infer the complete 3D geometry and semantic information of scenes from only 2D images. It has garnered significant attention, partic…