39 citations · 87 across the 19 of their papers we have counts for
21 papers
T2-z: Test-Time Optimization of Environment Latents for Robotic World Models
Chenyuan Zhou, Yining Hong, Huang Huang +9
Predicting how the world responds to an action requires more than what can be seen: robots must account for environmental factors that are neither directly observable nor explicitl…
DriveReferee: Geometric Safety Verdicts Need Not Be Learned for Driving World-Action Models
Fengcheng Yu, Dhruv Parikh, Junjie Ye +5
Generative world-action models (WAMs) jointly generate future video and vehicle actions, while their action branches remain primarily optimized by expert imitation. Yet imitation p…
Logic-VLA: A Temporal Logic Conditioned Vision-Language-Action Model
Celina Shiyu Wang, Yiqi Zhao, Junjie Ye +2
Vision-language-action (VLA) models can follow natural-language (NL) task instructions, but such instructions may not precisely specify safety-critical or spatiotemporal requiremen…
SIMPLE: Simulation-Based Policy Learning and Evaluation for Humanoid Loco-manipulation
Songlin Wei, Zhenhao Ni, Jie Liu +9
Humanoid foundation models are advancing faster than we can evaluate them. While real-world testing is expensive and difficult to reproduce, existing simulation benchmarks focus pr…
Theory of Space: Can Foundation Models Construct Spatial Beliefs through Active Exploration?
Pingyue Zhang, Zihan Huang, Yue Wang +11
Spatial embodied intelligence requires agents to act to acquire information under partial observability. While multimodal foundation models excel at passive perception, their capac…
PolaRiS: Scalable Real-to-Sim Evaluations for Generalist Robot Policies
Arhan Jain, Mingtong Zhang, Kanav Arora +11
A significant challenge for robot learning research is our ability to accurately measure and compare the performance of robot policies. Benchmarking in robotics is historically cha…
Robot Learning from a Physical World Model
Jiageng Mao, Sicheng He, Hao-Ning Wu +9
We introduce PhysWorld, a framework that enables robot learning from video generation through physical world modeling. Recent video generation models can synthesize photorealistic…
ROVER: Benchmarking Reciprocal Cross-Modal Reasoning for Omnimodal Generation
Yongyuan Liang, Wei Chow, Feng Li +7
Unified multimodal models (UMMs) have emerged as a powerful paradigm for seamlessly unifying text and image understanding and generation. However, prevailing evaluations treat thes…
Robot Learning from Any Images
Siheng Zhao, Jiageng Mao, Wei Chow +11
We introduce RoLA, a framework that transforms any in-the-wild image into an interactive, physics-enabled robotic environment. Unlike previous methods, RoLA operates directly on a…
Sensing, Social, and Motion Intelligence in Embodied Navigation: A Comprehensive Survey
Chaoran Xiong, Yulong Huang, Fangwen Yu +4
Embodied navigation (EN) advances traditional navigation by enabling robots to perform complex egocentric tasks through sensing, social, and motion intelligence. In contrast to cla…
ManipBench: Benchmarking Vision-Language Models for Low-Level Robot Manipulation
Enyu Zhao, Vedant Raval, Hejia Zhang +5
Vision-Language Models (VLMs) have revolutionized artificial intelligence and robotics due to their commonsense reasoning capabilities. In robotic manipulation, VLMs are used prima…
Domain-Conditioned Scene Graphs for State-Grounded Task Planning
Jonas Herzog, Jiangpin Liu, Yue Wang
Recent robotic task planning frameworks have integrated large multimodal models (LMMs) such as GPT-4o. To address grounding issues of such models, it has been suggested to split th…
UrbanVideo-Bench: Benchmarking Vision-Language Models on Embodied Intelligence with Video Data in Urban Spaces
Baining Zhao, Jianjie Fang, Zichao Dai +8
Large multimodal models exhibit remarkable intelligence, yet their embodied cognitive abilities during motion in open-ended urban 3D space remain to be explored. We introduce a ben…
PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understanding
Wei Chow, Jiageng Mao, Boyi Li +3
Understanding the physical world is a fundamental challenge in embodied AI, critical for enabling agents to perform complex tasks and operate safely in real-world environments. Whi…
Learning from Massive Human Videos for Universal Humanoid Pose Control
Jiageng Mao, Siheng Zhao, Siqi Song +7
Scalable learning of humanoid robots is crucial for their deployment in real-world applications. While traditional approaches primarily rely on reinforcement learning or teleoperat…
LiLoc: Lifelong Localization using Adaptive Submap Joining and Egocentric Factor Graph
Yixin Fang, Yanyan Li, Kun Qian +3
This paper proposes a versatile graph-based lifelong localization framework, LiLoc, which enhances its timeliness by maintaining a single central session while improves the accurac…
RAM: Retrieval-Based Affordance Transfer for Generalizable Zero-Shot Robotic Manipulation
Yuxuan Kuang, Junjie Ye, Haoran Geng +5
This work proposes a retrieve-and-transfer framework for zero-shot robotic manipulation, dubbed RAM, featuring generalizability across various objects, environments, and embodiment…
Driving Everywhere with Large Language Model Policy Adaptation
Boyi Li, Yue Wang, Jiageng Mao +4
Adapting driving behavior to new environments, customs, and laws is a long-standing problem in autonomous driving, precluding the widespread deployment of autonomous vehicles (AVs)…
A Language Agent for Autonomous Driving
Jiageng Mao, Junjie Ye, Yuxi Qian +2
Human-level driving is an ultimate goal of autonomous driving. Conventional approaches formulate autonomous driving as a perception-prediction-planning framework, yet their systems…
GPT-Driver: Learning to Drive with GPT
Jiageng Mao, Yuxi Qian, Junjie Ye +2
We present a simple yet effective approach that can transform the OpenAI GPT-3.5 model into a reliable motion planner for autonomous vehicles. Motion planning is a core challenge i…
SSCBench: A Large-Scale 3D Semantic Scene Completion Benchmark for Autonomous Driving
Yiming Li, Sihang Li, Xinhao Liu +11
Monocular scene understanding is a foundational component of autonomous systems. Within the spectrum of monocular perception topics, one crucial and useful task for holistic 3D sce…