9 papers
Embodied Agents Take Control: Minimal-Interface Zero-Shot Agents Rival Industrial-Scale Policies in Vision-and-Language Navigation
Jian Zhou, Xunyi Zhao, Gengze Zhou +4
Autonomous embodied agents must sustain a long decision-making loop of perceiving, acting, verifying, and self-correcting. In trained end-to-end policies and hand-designed workflow…
Ask When It Pays: Cost-Aware Open-Ended Interaction for Instance Goal Navigation
Xunyi Zhao, Sihao Lin, Gengze Zhou +5
Instance Goal Navigation (IGN) requires an embodied agent to find a specific object instance among distractors from an under-specified natural-language description. Such ambiguity…
TTT-VLA: Test-Time Latent Prompt Optimization for Vision-Language-Action Models
Wenbo Zhang, Jianxiong Li, Shuai Yang +4
Vision-Language-Action (VLA) models trained on large-scale data have made remarkable progress, but they remain vulnerable to distribution shifts at deployment time. Recent VLA mode…
VLNVerse: A Benchmark for Vision-Language Navigation with Versatile, Embodied, Realistic Simulation and Evaluation
Sihao Lin, Zerui Li, Xunyi Zhao +10
Despite remarkable progress in Vision-Language Navigation (VLN), existing benchmarks remain confined to fixed, small-scale datasets with naive physical simulation. These shortcomin…
The Singapore Consensus on Global AI Safety Research Priorities
Yoshua Bengio, Tegan Maharaj, Luke Ong +84
Rapidly improving AI capabilities and autonomy hold significant promise of transformation, but are also driving vigorous debate on how to ensure that AI is safe, i.e., trustworthy,…
Chain-of-Action: Trajectory Autoregressive Modeling for Robotic Manipulation
Wenbo Zhang, Tianrun Hu, Hanbo Zhang +7
We present Chain-of-Action (CoA), a novel visuo-motor policy paradigm built upon Trajectory Autoregressive Modeling. Unlike conventional approaches that predict next step action(s)…
General Scene Adaptation for Vision-and-Language Navigation
Haodong Hong, Yanyuan Qiao, Sen Wang +2
Vision-and-Language Navigation (VLN) tasks mainly evaluate agents based on one-time execution of individual instructions across multiple environments, aiming to develop agents capa…
RAGulator: Lightweight Out-of-Context Detectors for Grounded Text Generation
Ian Poey, Jiajun Liu, Qishuai Zhong +1
Real-time detection of out-of-context LLM outputs is crucial for enterprises looking to safely adopt RAG applications. In this work, we train lightweight models to discriminate LLM…
Effective Tuning Strategies for Generalist Robot Manipulation Policies
Wenbo Zhang, Yang Li, Yanyuan Qiao +5
Generalist robot manipulation policies (GMPs) have the potential to generalize across a wide range of tasks, devices, and environments. However, existing policies continue to strug…