21 papers
Orbia: Do Generated Videos Form a Consistent 3D World?
Dongyue Lu, Guian Fang, Hur Lim +20
Video generators serve as interactive worlds for exploration and simulation, yet realistic and visually appealing frames do not necessarily depict a consistent 3D world. Existing b…
VITAL: From Visual Motion to Precise Action
Ao Liang, Youquan Liu, Mingxiao Li +7
World-action models aim to turn visual anticipation into driving decisions, yet typically train action experts on trajectory representations defined independently of visual dynamic…
EventDrive: Event Cameras for Vision-Language Driving Intelligence
Dongyue Lu, Rong Li, Ao Liang +6
Event cameras sense the world through asynchronous brightness changes with microsecond latency and high dynamic range, offering motion fidelity far beyond frame-based sensors and c…
OmniLiDAR: A Unified Diffusion Framework for Multi-Domain 3D LiDAR Generation
Youquan Liu, Weidong Yang, Ao Liang +9
LiDAR scene generation is increasingly important for scalable simulation and synthetic data creation, especially under diverse sensing conditions that are costly to capture at scal…
Is Your Driving World Model an All-Around Player?
Lingdong Kong, Ao Liang, Tianyi Yan +20
Today's driving world models can generate remarkably realistic dash-cam videos, yet no single model excels universally. Some generate photorealistic textures but violate basic phys…
NavThinker: Action-Conditioned World Models for Coupled Prediction and Planning in Social Navigation
Tianshuai Hu, Zeying Gong, Lingdong Kong +7
Social navigation requires robots to act safely in dynamic human environments. Effective behavior demands thinking ahead: reasoning about how the scene and pedestrians evolve under…
The RoboSense Challenge: Sense Anything, Navigate Anywhere, Adapt Across Platforms
Lingdong Kong, Shaoyuan Xie, Zeying Gong +135
Autonomous systems are increasingly deployed in open and dynamic environments -- from city streets to aerial and indoor spaces -- where perception models must remain reliable under…
Vision-Language-Action Models for Autonomous Driving: Past, Present, and Future
Tianshuai Hu, Xiaolu Liu, Song Wang +17
Autonomous driving has long relied on modular "Perception-Decision-Action" pipelines, where hand-crafted interfaces and rule-based components often break down in complex or long-ta…
Learning to Remove Lens Flare in Event Camera
Haiqian Han, Lingdong Kong, Jianing Li +7
Event cameras have the potential to revolutionize vision systems with their high temporal resolution and dynamic range, yet they remain susceptible to lens flare, a fundamental opt…
WorldLens: Full-Spectrum Evaluations of Driving World Models in Real World
Ao Liang, Lingdong Kong, Tianyi Yan +19
Generative world models are reshaping embodied AI, enabling agents to synthesize realistic 4D driving environments that look convincing but often fail physically or behaviorally. D…
U4D: Uncertainty-Aware 4D World Modeling from LiDAR Sequences
Xiang Xu, Alan Liang, Youquan Liu +4
Modeling dynamic 3D environments from LiDAR sequences is central to building reliable 4D worlds for autonomous driving and embodied AI. Existing generative frameworks, however, oft…
3EED: Ground Everything Everywhere in 3D
Rong Li, Yuhao Dong, Tianshuai Hu +7
Visual grounding in 3D is the key for embodied agents to localize language-referred objects in open-world environments. However, existing benchmarks are limited to indoor focus, si…
See4D: Pose-Free 4D Generation via Auto-Regressive Video Inpainting
Dongyue Lu, Ao Liang, Tianxin Huang +8
Immersive applications call for synthesizing spatiotemporal 4D content from casual videos without costly 3D supervision. Existing video-to-4D methods typically rely on manually ann…
Learning to Generate 4D LiDAR Sequences
Ao Liang, Youquan Liu, Yu Yang +5
While generative world models have advanced video and occupancy-based data synthesis, LiDAR generation remains underexplored despite its importance for accurate 3D perception. Exte…
Visual Grounding from Event Cameras
Lingdong Kong, Dongyue Lu, Ao Liang +6
Event cameras capture changes in brightness with microsecond precision and remain reliable under motion blur and challenging illumination, offering clear advantages for modeling hi…
3D and 4D World Modeling: A Survey
Lingdong Kong, Yu Yang, Jianbiao Mei +20
World modeling has become a cornerstone in AI research, enabling agents to understand, represent, and predict the dynamic environments they inhabit. While prior work largely emphas…
La La LiDAR: Large-Scale Layout Generation from LiDAR Data
Youquan Liu, Lingdong Kong, Weidong Yang +5
Controllable generation of realistic LiDAR scenes is crucial for applications such as autonomous driving and robotics. While recent diffusion-based models achieve high-fidelity LiD…
Veila: Panoramic LiDAR Generation from a Monocular RGB Image
Youquan Liu, Lingdong Kong, Weidong Yang +8
Realistic and controllable panoramic LiDAR data generation is critical for scalable 3D perception in autonomous driving and robotics. Existing methods either perform unconditional…
LiDARCrafter: Dynamic 4D World Modeling from LiDAR Sequences
Ao Liang, Youquan Liu, Yu Yang +5
Generative world models have become essential data engines for autonomous driving, yet most existing efforts focus on videos or occupancy grids, overlooking the unique LiDAR proper…
Perspective-Invariant 3D Object Detection
Ao Liang, Lingdong Kong, Dongyue Lu +4
With the rise of robotics, LiDAR-based 3D object detection has garnered significant attention in both academia and industry. However, existing datasets and methods predominantly fo…
Talk2Event: Grounded Understanding of Dynamic Scenes from Event Cameras
Lingdong Kong, Dongyue Lu, Ao Liang +6
Event cameras offer microsecond-level latency and robustness to motion blur, making them ideal for understanding dynamic environments. Yet, connecting these asynchronous streams to…