16 papers
DynamicWAM: Dual-Path Motion Conditioning for World-Action Models in Dynamic Manipulation
Yunfan Lou, Hewen Gao, Xiyu Zhu +6
Dynamic manipulation requires robots to infer target motion and respond promptly, yet existing World-Action Models (WAMs) typically condition only on the current frame and execute…
Open-AoE: An Open Egocentric Manipulation Dataset and Toolchain for Embodied Learning
Zishuo Li, Bowen Yang, Changtao Miao +29
Egocentric videos of human manipulation provide scalable supervision for embodied intelligence, yet existing resources rarely combine low-cost continuous capture, manipulation-leve…
VGGT-Edit: Feed-forward Native 3D Scene Editing with Residual Field Prediction
Kaixin Zhu, Yiwen Tang, Yifan Yang +9
High-quality 3D scene reconstruction has recently advanced toward generalizable feed-forward architectures, enabling the generation of complex environments in a single forward pass…
MemCompiler: Compile, Don't Inject -- State-Conditioned Memory for Embodied Agents
Xin Ding, Xinrui Wang, Yifan Yang +9
Existing memory systems for embodied agents typically inject retrieved memory as static context at episode start, a paradigm we term Ahead-of-time Monolithic Memory Injection (AMMI…
Covering Human Action Space for Computer Use: Data Synthesis and Benchmark
Miaosen Zhang, Xiaohan Zhao, Zhihong Tan +14
Computer-use agents (CUAs) automate on-screen work, as illustrated by GPT-5.4 and Claude. Yet their reliability on complex, low-frequency interactions is still poor, limiting user…
Geometry Conflict: Explaining and Controlling Forgetting in LLM Continual Post-Training
Yuanyi Wang, Yifan Yang, Su Lu +9
Continual post-training aims to extend large language models (LLMs) with new knowledge, skills, and behaviors, yet it remains unclear when sequential updates enable capability tran…
Evaluating the Expressive Appropriateness of Speech in Rich Contexts
Tianrui Wang, Ziyang Ma, Yizhou Peng +26
Evaluating expressive speech remains challenging, as existing methods mainly assess emotional intensity and overlook whether a speech sample is expressively appropriate for its con…
EmbodiSkill: Skill-Aware Reflection for Self-Evolving Embodied Agents
Ruofei Ju, Xinrui Wang, Xin Ding +12
Embodied agents can benefit from skills that guide object search, action execution, and state changes across diverse environments. Since embodied environments vary across layouts,…
AoE: Always-on Egocentric Human Video Collection for Embodied AI
Bowen Yang, Zishuo Li, Yang Sun +15
Embodied foundation models require large-scale, high-quality real-world interaction data for pre-training and scaling. However, existing data collection methods suffer from high in…
Em-Garde: A Propose-Match Framework for Proactive Streaming Video Understanding
Yikai Zheng, Xin Ding, Yifan Yang +6
Recent advances in Streaming Video Understanding has enabled a new interaction paradigm where models respond proactively to user queries. Current proactive VideoLLMs rely on per-fr…
AdaNav: Adaptive Reasoning with Uncertainty for Vision-Language Navigation
Xin Ding, Jianyu Wei, Yifan Yang +10
Vision Language Navigation (VLN) requires agents to follow natural language instructions by grounding them in sequential visual observations over long horizons. Explicit reasoning…
Zoomer: Adaptive Image Focus Optimization for Black-box MLLM
Jiaxu Qian, Chendong Wang, Yifan Yang +16
Multimodal large language models (MLLMs) such as GPT-4o, Gemini Pro, and Claude 3.5 have enabled unified reasoning over text and visual inputs, yet they often hallucinate in real w…
AVA: Towards Agentic Video Analytics with Vision Language Models
Yuxuan Yan, Shiqi Jiang, Ting Cao +5
AI-driven video analytics has become increasingly important across diverse domains. However, existing systems are often constrained to specific, predefined tasks, limiting their ad…
StreamMind: Unlocking Full Frame Rate Streaming Video Dialogue through Event-Gated Cognition
Xin Ding, Hao Wu, Yifan Yang +4
With the rise of real-world human-AI interaction applications, such as AI assistants, the need for Streaming Video Dialogue is critical. To address this need, we introduce StreamMi…
Making Every Frame Matter: Continuous Activity Recognition in Streaming Video via Adaptive Video Context Modeling
Hao Wu, Donglin Bai, Shiqi Jiang +6
Video activity recognition has become increasingly important in robots and embodied AI. Recognizing continuous video activities poses considerable challenges due to the fast expans…
Babel: A Scalable Pre-trained Model for Multi-Modal Sensing via Expandable Modality Alignment
Shenghong Dai, Shiqi Jiang, Yifan Yang +4
This paper presents Babel, the expandable modality alignment model, specially designed for multi-modal sensing. While there has been considerable work on multi-modality alignment,…