8 papers
From Routes to Steps: Separating Semantic Progress from Local Execution in Vision-and-Language Navigation
Xiangyun Huang, Xiangchen Wang, Runfeng Lin +5
Vision-and-Language Navigation (VLN) requires an agent to follow a route-level instruction by executing its constituent steps from egocentric visual observations. Existing VLM-base…
CAPruner: Conceptual-Adjacent Scene Graph Pruner for Enhancing 3D Spatial Reasoning of Large Language Models
Shengli Zhou, Xiangchen Wang, Guanhua Chen +1
Large language models (LLMs) have recently been applied to 3D vision-language (3D-VL) tasks, which require spatial reasoning to identify target objects relative to anchors. Scene g…
LiveVLN: Breaking the Stop-and-Go Loop in Vision-Language Navigation
Xiangchen Wang, Weiye Zhu, Teng Wang +5
Recent navigation systems achieve strong benchmark results, yet real-world deployment often remains visibly stop-and-go. This bottleneck arises because the sense-inference-executio…
\textsc{NaVIDA}: Vision-Language Navigation with Inverse Dynamics Augmentation
Weiye Zhu, Zekai Zhang, Xiangchen Wang +5
Vision-and-Language Navigation (VLN) requires agents to interpret natural language instructions and act coherently in visually rich environments. However, most existing methods rel…
ActiveVLN: Towards Active Exploration via Multi-Turn RL in Vision-and-Language Navigation
Zekai Zhang, Weiye Zhu, Hewei Pan +4
The Vision-and-Language Navigation (VLN) task requires an agent to follow natural language instructions and navigate through complex environments. Existing MLLM-based VLN methods p…
Seeing More, Saying More: Lightweight Language Experts are Dynamic Video Token Compressors
Xiangchen Wang, Jinrui Zhang, Teng Wang +2
Recent advancements in large video-language models have revolutionized video understanding tasks. However, their efficiency is significantly constrained by processing high volumes…
: Toward Versatile Embodied Agents
Shengli Zhou, Xiangchen Wang, Jinrui Zhang +5
Embodied agents have demonstrated promising capabilities in interacting with physical environments. Yet, versatile embodied agents face three core bottlenecks: dynamic environmenta…
LLMVA-GEBC: Large Language Model with Video Adapter for Generic Event Boundary Captioning
Yolo Yunlong Tang, Jinrui Zhang, Xiangchen Wang +2
Our winning entry for the CVPR 2023 Generic Event Boundary Captioning (GEBC) competition is detailed in this paper. Unlike conventional video captioning tasks, GEBC demands that th…