9 papers
RxBrain: Embodied Cognition Foundation Model with Joint Language-Visual Reasoning and Imagination
Haotian Liang, Mingkang Chen, Yufei Huang +27
Embodied cognition requires agents to connect high-level task reasoning with the physical states to be achieved. We introduce Hy-Embodied-RxBrain, an embodied cognition foundation…
Listening with the Eyes: Benchmarking Egocentric Co-Speech Grounding across Space and Time
Weijie Zhou, Xuantang Xiong, Zhenlin Hu +6
In situated collaboration, speakers often use intentionally underspecified deictic commands (e.g., ``pass me \textit{that}''), whose referent becomes identifiable only by aligning…
ProAct: A Benchmark and Multimodal Framework for Structure-Aware Proactive Response
Xiaomeng Zhu, Fengming Zhu, Weijie Zhou +8
While passive agents merely follow instructions, proactive agents align with higher-level objectives, such as assistance and safety by continuously monitoring the environment to de…
SteerEval: A Framework for Evaluating Steerability with Natural Language Profiles for Recommendation
Joyce Zhou, Weijie Zhou, Doug Turnbull +1
Natural-language user profiles have recently attracted attention not only for improved interpretability, but also for their potential to make recommender systems more steerable. By…
UniBYD: A Unified Framework for Learning Robotic Manipulation Across Embodiments Beyond Imitation of Human Demonstrations
Tingyu Yuan, Biaoliang Guan, Wen Ye +8
In embodied intelligence, the embodiment gap between robotic and human hands brings significant challenges for learning from human demonstrations. Although some studies have attemp…
PhysVLM-AVR: Active Visual Reasoning for Multimodal Large Language Models in Physical Environments
Weijie Zhou, Xuantang Xiong, Yi Peng +5
Visual reasoning in multimodal large language models (MLLMs) has primarily been studied in static, fully observable settings, limiting their effectiveness in real-world environment…
PhysVLM: Enabling Visual Language Models to Understand Robotic Physical Reachability
Weijie Zhou, Manli Tao, Chaoyang Zhao +4
Understanding the environment and a robot's physical reachability is crucial for task execution. While state-of-the-art vision-language models (VLMs) excel in environmental percept…
LightPlanner: Unleashing the Reasoning Capabilities of Lightweight Large Language Models in Task Planning
Weijie Zhou, Manli Tao, Chaoyang Zhao +3
In recent years, lightweight large language models (LLMs) have garnered significant attention in the robotics field due to their low computational resource requirements and suitabi…
Discrete Messages Improve Communication Efficiency among Isolated Intelligent Agents
Hang Chen, Yuchuan Jang, Weijie Zhou +3
Individuals, despite having varied life experiences and learning processes, can communicate effectively through languages. This study aims to explore the efficiency of language as…