1 citations · 2 across the 46 of their papers we have counts for
54 papers
EVOKE: Eliciting World Knowledge in Agents for Transferable Decision-Making
Yuhan Guo, Jinming Liu, Liang Xu +8
Large language models (LLMs) are increasingly deployed as agents for multi-step decision-making, yet transfer poorly to unseen environments. World-model methods address this by tra…
APM-Bench: Benchmarking Cross-session Persistent Memory for Egocentric Streaming Video Assistants
Jianguo Huang, Jinming Liu, Qiyao Wang +9
To serve as real-world personal assistants, streaming video models need persistent memory that retains past experiences for later use. Yet existing streaming benchmarks and methods…
FAN: Foresight Action Normalization for Continual Adaptation of Vision-Language-Action Models
Yijun Hong, Jiarun Zhu, Xiaoquan Sun +6
Vision-Language-Action (VLA) models pre-trained on large-scale, closed datasets have demonstrated remarkable success across diverse robotic manipulation tasks. However, their long-…
Inter-X++: A Comprehensive Benchmark for Multimodal Human-Human Interaction Analysis
Liang Xu, Chengqun Yang, Zili Lin +6
The capability to perceive and synthesize human-human interactions is fundamental to developing intelligent digital human systems. However, existing datasets and modeling approache…
MMOOC: A Comprehensive Benchmark for Out-of-Context Evaluation in Multimodal Large Language Models
Wenjie Zhu, Yabin Zhang, Wenjun Zeng +1
Multimodal Large Language Models (MLLMs) have achieved strong performance on a wide range of vision-language tasks, but often fail under imperfect or shifted contexts. A reliable M…
GEM-Occ: From Visual Geometry Evidence to Embodied Semantic Occupancy Memory
Hu Zhu, Bohan Li, Xianda Guo +8
Embodied agents exploring indoor environments require reliable semantic occupancy memory that persists across observations and revisits. Building such memory is challenging because…
Bridging 3D Gaussians and Semantic Occupancy for Comprehensive Open-Vocabulary Scene Understanding from Unposed Images
Hu Zhu, Bohan Li, Xianda Guo +5
Comprehensive 3D scene understanding from sparse, unposed images requires a model to recover renderable geometry, open-vocabulary semantics, and free/occupied 3D space without rely…
DeformGen: Dynamics-Based Topology Augmentation for Deformable Manipulation Policy Learning
Zili Lin, Wenyao Zhang, Yuyang Zhang +9
Demonstration augmentation is proposed for cost-efficient data acquisition, but existing methods are fundamentally limited in deformable manipulation due to two challenges: (1) the…
ImageWAM: Do World Action Models Really Need Video Generation, or Just Image Editing?
Yuyang Zhang, Wenyao Zhang, Zekun Qi +7
World Action Models (WAMs) commonly rely on video generation to bridge visual world modeling and robot control. However, video-based WAMs face three coupled limitations: dense mult…
Dual Distribution Estimation for Zero-shot Noisy Test-Time Adaptation with VLMs
Wenjie Zhu, Yabin Zhang, Liang Xu +3
While test-time adaptation (TTA) empowers vision-language models to adapt without costly retraining, it remains highly vulnerable to out-of-distribution (OOD) outliers prevalent in…
An Efficient Streaming Video Understanding Framework with Agentic Control
Jinming Liu, Jianguo Huang, Zhaoyang Jia +7
Streaming video requires handling dynamic information density under strict latency budgets. Yet, existing methods typically employ static strategies, such as fixed memory compressi…
Generation Navigator: A State-Aware Agentic Framework for Image Generation
Jinming Liu, Ruoyu Feng, Yuqi Wang +2
Despite rapid advances in text-to-image generation, faithfully realizing user intent remains challenging, often requiring manual multi-turn trial and error. To automate this proces…
From Articulated Kinematics to Routed Visual Control for Action-Conditioned Surgical Video Generation
Bohan Li, Shuojue Yang, Baorui Peng +10
Action-conditioned surgical video generation is a critical yet highly challenging problem for robotic surgery. The core difficulty is that low-dimensional control vectors must prec…
SceneScribe-1M: A Large-Scale Video Dataset with Comprehensive Geometric and Semantic Annotations
Yunnan Wang, Kecheng Zheng, Jianyuan Wang +8
The convergence of 3D geometric perception and video synthesis has created an unprecedented demand for large-scale video data that is rich in both semantic and spatio-temporal info…
Disentangled Robot Learning via Separate Forward and Inverse Dynamics Pretraining
Wenyao Zhang, Bozhou Zhang, Zekun Qi +3
Vision-language-action (VLA) models have shown great potential in building generalist robots, but still face a dilemma-misalignment of 2D image forecasting and 3D action prediction…
Compression Tells Intelligence: Visual Coding, Visual Token Technology, and the Unification
Xin Jin, Jinming Liu, Yuntao Wei +6
"Compression Tells Intelligence", is supported by research in artificial intelligence, particularly concerning (multimodal) large language models (LLMs/MLLMs), where compression ef…
ReWorld: Multi-Dimensional Reward Modeling for Embodied World Models
Baorui Peng, Wenyao Zhang, Liang Xu +5
Recently, video-based world models that learn to simulate the dynamics have gained increasing attention in robot learning. However, current approaches primarily emphasize visual ge…
PvP: Data-Efficient Humanoid Robot Learning with Proprioceptive-Privileged Contrastive Representations
Mingqi Yuan, Tao Yu, Haolin Song +4
Achieving efficient and robust whole-body control (WBC) is essential for enabling humanoid robots to perform complex tasks in dynamic environments. Despite the success of reinforce…
Goal-Driven Reward by Video Diffusion Models for Reinforcement Learning
Qi Wang, Mian Wu, Yuyang Zhang +7
Reinforcement Learning (RL) has achieved remarkable success in various domains, yet it often relies on carefully designed programmatic reward functions to guide agent behavior. Des…
Vision-Centric Activation and Coordination for Multimodal Large Language Models
Yunnan Wang, Fan Lu, Kecheng Zheng +4
Multimodal large language models (MLLMs) integrate image features from visual encoders with LLMs, demonstrating advanced comprehension capabilities. However, mainstream MLLMs are s…
Hybrid-grained Feature Aggregation with Coarse-to-fine Language Guidance for Self-supervised Monocular Depth Estimation
Wenyao Zhang, Hongsi Liu, Bohan Li +7
Current self-supervised monocular depth estimation (MDE) approaches encounter performance limitations due to insufficient semantic-spatial knowledge extraction. To address this cha…
Scaling Up Occupancy-centric Driving Scene Generation: Dataset and Method
Bohan Li, Xin Jin, Hu Zhu +9
Driving scene generation is a critical domain for autonomous driving, enabling downstream applications, including perception and planning evaluation. Occupancy-centric methods have…
OmniNWM: Omniscient Driving Navigation World Models
Bohan Li, Zhuang Ma, Dalong Du +10
Autonomous driving world models are expected to work effectively across three core dimensions: state, action, and reward. However, existing methods are typically restricted to frag…
When MLLMs Meet Compression Distortion: A Coding Paradigm Tailored to MLLMs
Jinming Liu, Zhaoyang Jia, Jiahao Li +4
The increasing deployment of powerful Multimodal Large Language Models (MLLMs), typically hosted on cloud platforms, urgently requires effective compression techniques to efficient…