14 papers
Failure Recovery via Belief Updates and World Modeling for Long-Horizon Agents
Xiyang Wu, Zhichao Liu, Shengxin zhang +4
Long-horizon agents often struggle to recover from failures whose causes lie several steps earlier, beyond what local reflection or backtracking can resolve. We introduce BELIEFWEA…
Causeway: Restoring Task Accessibility for Instruction Switching in VLA Policies
Qingzi Wang, Kaixi Feng, Guangyao Shi +3
Vision-language-action (VLA) policies can execute many tasks from standard initial states, yet a new instruction may fail after another task has altered the robot's physical state.…
VEGA: Learning Navigation VLAs from In-the-Wild Egocentric Video with Geometric Trajectory Supervision
Gershom Seneviratne, Yohan Abeysinghe, Jianyu An +3
We introduce VEGA, an approach for training navigation VisionLanguage-Action (VLA) models from unlabeled egocentric navigation videos. Internet-scale egocentric videos provide a sc…
A Closer Look at Failure Modes in Temporal Understanding of Large Audio-Language Models
Apoorva Kulkarni, Kaousheik Jayakumar, Sreyan Ghosh +3
Large Audio Language Models (LALMs) achieve strong performance on a variety of audio understanding tasks but continue to struggle with temporal reasoning, a fundamental capability…
Act on What You See: Unlocking Safe Social Navigation in Vision-Language-Action Models
Qingzi Wang, Xiyang Wu, Guangyao Shi +3
Safe social navigation requires robots to distinguish people from ordinary obstacles and to react before danger becomes imminent. We show that pretrained Vision-Language-Action (VL…
Co-Evolving LLM Decision and Skill Bank Agents for Long-Horizon Tasks
Xiyang Wu, Zongxia Li, Guangyao Shi +5
Long horizon interactive environments are a testbed for evaluating agents skill usage abilities. These environments demand multi step reasoning, the chaining of multiple skills ove…
SABER: A Stealthy Agentic Black-Box Attack Framework for Vision-Language-Action Models
Xiyang Wu, Guangyao Shi, Qingzi Wang +3
Vision-language-action (VLA) models enable robots to follow natural-language instructions grounded in visual observations, but the instruction channel also introduces a critical vu…
TransCurriculum: Multi-Dimensional Curriculum Learning for Fast & Stable Locomotion
Prakhar Mishra, Amir Hossain Raj, Xuesu Xiao +1
High-speed legged locomotion struggles with stability and transfer losses at higher command velocities during deployment. One reason is that most curricula vary difficulty along si…
CHOP: Counterfactual Human Preference Labels Improve Obstacle Avoidance in Visuomotor Navigation Policies
Gershom Seneviratne, Jianyu An, Vaibhav Shende +6
Visuomotor navigation policies have shown strong perception-action coupling for embodied agents, yet they often struggle with safe navigation and dynamic obstacle avoidance in comp…
MorFiC: Fixing Value Miscalibration for Zero-Shot Quadruped Transfer
Prakhar Mishra, Amir Hossain Raj, Xuesu Xiao +1
Generalizing learned locomotion policies across quadrupedal robots with different morphologies remains a challenge. Policies trained on a single robot often fail when deployed on e…
MMOU: A Massive Multi-Task Omni Understanding and Reasoning Benchmark for Long and Complex Real-World Videos
Arushi Goel, Sreyan Ghosh, Vatsal Agarwal +16
Multimodal Large Language Models (MLLMs) have shown strong performance in visual and audio understanding when evaluated in isolation. However, their ability to jointly reason over…
EgoSocial: Benchmarking Proactive Intervention Ability of Omnimodal LLMs via Egocentric Social Interaction Perception
Xijun Wang, Tanay Sharma, Achin Kulshrestha +3
As AR/VR technologies become integral to daily life, there's a growing need for AI that understands human social dynamics from an egocentric perspective. However, current LLMs ofte…
McARL:Morphology-Control-Aware Reinforcement Learning for Generalizable Quadrupedal Locomotion
Prakhar Mishra, Amir Hossain Raj, Xuesu Xiao +1
We present Morphology-Control-Aware Reinforcement Learning (McARL), a new approach to overcome challenges of hyperparameter tuning and transfer loss, enabling generalizable locomot…
HACL: History-Aware Curriculum Learning for Fast Locomotion
Prakhar Mishra, Amir Hossain Raj, Xuesu Xiao +1
We address the problem of agile and rapid locomotion, a key characteristic of quadrupedal and bipedal robots. We present a new algorithm that maintains stability and generates high…