9 papers
HM3D-M3: A Benchmark for Multi-Agent, Multi-Modal, Multi-Object Goal Navigation
Hongyu Ding, Kanyu Xu, Mingjia Chen +5
Rapid progress in AI agents and robotics makes it a matter of the near future that humans are served by a team of robots. Such a team will receive several goals at once, each speci…
GeoScaffold: Learning Compact Geometric Latents via Reconstruction for Efficient Vision-Language Navigation
Yixuan Jiang, Wentong Li, An Liu +5
Recent vision-and-language navigation (VLN) systems increasingly adopt streaming Video-LLM policies that map egocentric RGB observations and instructions directly to low-level acti…
ReScene: Structured Indoor Scene Reconstruction from Multi-View Captures
Haoran Xu, Lechao Zhang, Daoguo Dong +2
Constructing simulation-ready 3D scenes from multi-view captures is a key bottleneck for Embodied Artificial Intelligence, as downstream tasks require object-level structure, expli…
Deformable Gaussian Occupancy: Decoupling Rigid and Nonrigid Motion with Factorized Distillation
Yang Gao, Wuyang Li, Po-Chien Luan +1
Understanding dynamic 3D environments is essential for safe autonomous driving, particularly when reasoning about human-centric, nonrigid agents. However, existing weakly supervise…
Uni-LaViRA: Language-Vision-Robot Actions Translation for Unified Embodied Navigation
Hongyu Ding, Sizhuo Zhang, Ziming Xu +13
Embodied navigation requires an agent to map language and visual observations to a stream of spatial actions that drive a real robot through environments it has never seen. The dom…
INHerit-SG: Incremental Hierarchical Semantic Scene Graphs for Complex Embodied Retrieval
YukTungSamuel Fang, Zhikang Shi, Jiabin Qiu +5
Object retrieval in 3D scenes remains difficult when queries combine hierarchical context, attributes, negation, and chained spatial relations. Existing semantic maps, scene graphs…
VITA-VLA: Efficiently Teaching Vision-Language Models to Act via Action Expert Distillation
Shaoqi Dong, Chaoyou Fu, Haihan Gao +12
Vision-Language Action (VLA) models significantly advance robotic manipulation by leveraging the strong perception capabilities of pretrained vision-language models (VLMs). By inte…
SEA: Semantic Map Prediction for Active Exploration of Uncertain Areas
Hongyu Ding, Xinyue Liang, Yudong Fang +7
In this paper, we propose SEA, a novel approach for active robot exploration through semantic map prediction and a reinforcement learning-based hierarchical exploration policy. Unl…
LaViRA: Language-Vision-Robot Actions Translation for Zero-Shot Vision Language Navigation in Continuous Environments
Hongyu Ding, Ziming Xu, Yudong Fang +6
LaViRA: Zero-shot Vision-and-Language Navigation in Continuous Environments (VLN-CE) requires an agent to navigate unseen environments based on natural language instructions withou…