1 citations · 2 across the 24 of their papers we have counts for
32 papers
TacitVLA: Reasoning Vision-Language-Action Models Without Reasoning Latency
Jack Lu, Yurong You, Boyi Li +8
Textual reasoning before acting has improved vision-language-action (VLA) models, but at the cost of latency: every action conditioned on the reasoning text must wait for autoregre…
SCOPD: Sparse-Context On-Policy Self-Distillation for Efficient Vision-Language Models
Ahmadreza Jeddi, Enming Zhang, Jasper Gerigk +12
Reasoning vision-language models (VLMs) process images and videos as long sequences of visual tokens, making inference expensive. Training-free token pruning reduces this cost, but…
A Scene Language Model for Open-Vocabulary Scene Mapping
Adam Lilja, Fabio Hübel, Siming He +6
Open-vocabulary 3D scene mapping aims to build a persistent representation of the objects in an environment. Existing systems typically rely on engineered mapping pipelines to asso…
FARM: Find Anything using Relational Spatial Memory
Siming He, Leo Huang, Adam Lilja +6
Robots operating in homes, warehouses, and other object-rich environments need memory systems that can find specific object instances on demand. Object-level memory alone is often…
Towards Spatial Supersensing in the Wild
Tianjun Gu, Tianyu Xin, Kuan Zhang +12
Humans can efficiently parse continuous sensory streams, from hours to years, scaffolding an internal world model that grounds spatial reasoning and prediction. To mimic this capac…
WildCity: A Real-World City-Scale Testbed for Rendering, Simulation, and Spatial Intelligence
Xiangyu Han, Mengyu Yang, Jiaqi Li +9
Humans can navigate an unfamiliar city and gradually form a coherent spatial mental map spanning tens of square kilometers. Can AI build spatial representations at a comparable sca…
Cosmos 3: Omnimodal World Models for Physical AI
NVIDIA, :, Aditi +293
We introduce Cosmos 3, a family of omnimodal world models designed to jointly process and generate language, image, video, audio, and action sequences within a unified mixture-of-t…
Planning-aligned Token Compression for Long-Context Autonomous Driving
Zhixuan Liang, Yuxiao Chen, Yurong You +12
Monolithic vision-action models represent an emerging paradigm in autonomous driving. However, this architecture produces token sequences that quickly exceed real-time computationa…
Fast-dDrive: Efficient Block-Diffusion VLM for Autonomous Driving
Kewei Zhang, Jin Wang, Sensen Gao +9
End-to-end autonomous driving via Vision-Language-Action (VLA) models demands a precarious balance between high-fidelity trajectory planning and efficient inference. Existing parad…
Bridging Structure and Language: Graph-Based Visual Reasoning for Autonomous Road Understanding
Lena Wild, Katie Z Luo, Marco Pavone
Structured road understanding of lane geometry, topology, and traffic element relationships is foundational to safe autonomous driving. While vision-language models (VLMs) offer pr…
VISTAQA: Benchmarking Joint Visual Question Answering and Pixel-Level Evidence
Mozhgan Nasr Azadani, Yimu Wang, Yongpeng Zhu +5
Establishing a clear link between model predictions and the visual evidence that supports them is critical for transparency and reliability in multimodal reasoning, yet current mul…
: An Open Foundation Model Towards Universal Humanoid Loco-Manipulation
Songlin Wei, Hongyi Jing, Boqian Li +12
We introduce (Psi-Zero), an open foundation model to address challenging humanoid loco-manipulation tasks. While existing approaches often attempt to address this fundamental…
Accelerating Structured Chain-of-Thought in Autonomous Vehicles
Yi Gu, Yan Wang, Yuxiao Chen +8
Chain-of-Thought (CoT) reasoning enhances the decision-making capabilities of vision-language-action models in autonomous driving, but its autoregressive nature introduces signific…
TGIF: Text-Guided Layer Fusion Mitigates Hallucination in Multimodal LLMs
Chenchen Lin, Sanbao Su, Rachel Luo +4
Multimodal large language models (MLLMs) typically rely on a single late-layer feature from a frozen vision encoder, leaving the encoder's rich hierarchy of visual cues under-utili…
Counterfactual VLA: Self-Reflective Vision-Language-Action Model with Adaptive Reasoning
Zhenghao "Mark" Peng, Wenhao Ding, Yurong You +11
Recent reasoning-augmented Vision-Language-Action (VLA) models have improved the interpretability of end-to-end autonomous driving by generating intermediate reasoning traces. Yet…
Towards Efficient and Effective Multi-Camera Encoding for End-to-End Driving
Jiawei Yang, Ziyu Chen, Yurong You +7
We present Flex, an efficient and effective scene encoder that addresses the computational bottleneck of processing high-volume multi-camera data in end-to-end autonomous driving.…
Thinking in 360°: Humanoid Visual Search in the Wild
Heyang Yu, Yinan Han, Xiangyu Zhang +9
Humans rely on the synergistic control of head (cephalomotor) and eye (oculomotor) to efficiently search for visual information in 360°. However, prior approaches to visual search…
Bias in Gender Bias Benchmarks: How Spurious Features Distort Evaluation
Yusuke Hirota, Ryo Hachiuma, Boyi Li +9
Gender bias in vision-language foundation models (VLMs) raises concerns about their safe deployment and is typically evaluated using benchmarks with gender annotations on real-worl…
AccidentBench: Benchmarking Multimodal Understanding and Reasoning in Vehicle Accidents and Beyond
Shangding Gu, Xiaohan Wang, Donghao Ying +9
Rapid advances in multimodal models demand benchmarks that rigorously evaluate understanding and reasoning in safety-critical, dynamic real-world settings. We present AccidentBench…
The Case for Negative Data: From Crash Reports to Counterfactuals for Reasonable Driving
Jay Patrikar, Apoorva Sharma, Sushant Veer +3
Learning-based autonomous driving systems are trained mostly on incident-free data, offering little guidance near safety-performance boundaries. Real crash reports contain precisel…
LOTUS: A Leaderboard for Detailed Image Captioning from Quality to Societal Bias and User Preferences
Yusuke Hirota, Boyi Li, Ryo Hachiuma +7
Large Vision-Language Models (LVLMs) have transformed image captioning, shifting from concise captions to detailed descriptions. We introduce LOTUS, a leaderboard for evaluating de…
Efficient Multi-Camera Tokenization with Triplanes for End-to-End Driving
Boris Ivanovic, Cristiano Saltori, Yurong You +3
Autoregressive Transformers are increasingly being deployed as end-to-end robot and autonomous vehicle (AV) policy architectures, owing to their scalability and potential to levera…
E3D-Bench: A Benchmark for End-to-End 3D Geometric Foundation Models
Wenyan Cong, Yiqing Liang, Yancheng Zhang +7
Spatial intelligence, encompassing 3D reconstruction, perception, and reasoning, is fundamental to applications such as robotics, aerial imaging, and extended reality. A key enable…
Martian World Model: Controllable Video Synthesis with Physically Accurate 3D Reconstructions
Longfei Li, Zhiwen Fan, Wenyan Cong +10
Synthesizing realistic Martian landscape videos is crucial for mission rehearsal and robotic simulation. However, this task poses unique challenges due to the scarcity of high-qual…