73 citations · 202 across the 39 of their papers we have counts for
40 papers
Tired Actor: Fatigue-Informed Character Control
Shengyuan Zhang, Xinpeng Liu, Muchun Niu +5
Replicating human behavior with physics simulation has been a long-expected goal in character animation. Existing efforts have achieved impressive performance in imitating a wide s…
ChronoFlow-Policy: Unifying Past-Current-Future Interaction Flow in Visuomotor Policy Learning
Bokai Lin, Yifu Xu, Xinyu Zhan +6
Visual signals play a crucial role in policy learning by enabling models to capture object motion and interaction dynamics. Just as humans reason about actions using both past expe…
LIDEA: Human-to-Robot Imitation Learning via Implicit Feature Distillation and Explicit Geometry Alignment
Yifu Xu, Bokai Lin, Xinyu Zhan +4
Scaling up robot learning is hindered by the scarcity of robotic demonstrations, whereas human videos offer a vast, untapped source of interaction data. However, bridging the embod…
L1 Sample Flow for Efficient Visuomotor Learning
Weixi Song, Zhetao Chen, Tao Xu +6
Denoising-based models, such as diffusion and flow matching, have been a critical component of robotic manipulation for their strong distribution-fitting and scaling capacity. Conc…
SIME: Enhancing Policy Self-Improvement with Modal-level Exploration
Yang Jin, Jun Lv, Wenye Yu +3
Self-improvement requires robotic systems to initially learn from human-provided data and then gradually enhance their capabilities through interaction with the environment. This i…
Dense Policy: Bidirectional Autoregressive Learning of Actions
Yue Su, Xinyu Zhan, Hongjie Fang +5
Mainstream visuomotor policies predominantly rely on generative models for holistic action prediction, while current autoregressive policies, predicting the next token or chunk, ha…
Interacted Object Grounding in Spatio-Temporal Human-Object Interactions
Xiaoyang Liu, Boran Wen, Xinpeng Liu +6
Spatio-temporal Human-Object Interaction (ST-HOI) understanding aims at detecting HOIs from videos, which is crucial for activity understanding. However, existing whole-body-object…
Homogeneous Dynamics Space for Heterogeneous Humans
Xinpeng Liu, Junxuan Liang, Chenshuo Zhang +3
Analyses of human motion kinematics have achieved tremendous advances. However, the production mechanism, known as human dynamics, is still undercovered. In this paper, we aim to p…
Verb Mirage: Unveiling and Assessing Verb Concept Hallucinations in Multimodal Large Language Models
Zehao Wang, Xinpeng Liu, Yudonglin Zhang +6
Multimodal Large Language Models (MLLMs) have garnered significant attention recently and demonstrate outstanding capabilities in various tasks such as OCR, VQA, captioning, $\text…
Motion Before Action: Diffusing Object Motion as Manipulation Condition
Yue Su, Xinyu Zhan, Hongjie Fang +3
Inferring object motion representations from observations enhances the performance of robotic manipulation tasks. This paper introduces a new paradigm for robot imitation learning…
ImDy: Human Inverse Dynamics from Imitated Observations
Xinpeng Liu, Junxuan Liang, Zili Lin +3
Inverse dynamics (ID), which aims at reproducing the driven torques from human kinematic observations, has been a critical tool for gait analysis. However, it is hindered from wide…
The Labyrinth of Links: Navigating the Associative Maze of Multi-modal LLMs
Hong Li, Nanxi Li, Yuanjie Chen +4
Multi-modal Large Language Models (MLLMs) have exhibited impressive capability. However, recently many deficiencies of MLLMs have been found compared to human intelligence, $\texti…
Take A Step Back: Rethinking the Two Stages in Visual Reasoning
Mingyu Zhang, Jiting Cai, Mingyu Liu +3
Visual reasoning, as a prominent research area, plays a crucial role in AI by facilitating concept formation and interaction with the world. However, current works are usually carr…
DISCO: Embodied Navigation and Interaction via Differentiable Scene Semantics and Dual-level Control
Xinyu Xu, Shengcheng Luo, Yanchao Yang +2
Building a general-purpose intelligent home-assistant agent skilled in diverse tasks by human commands is a long-term blueprint of embodied AI research, which poses requirements on…
Human-Agent Joint Learning for Efficient Robot Manipulation Skill Acquisition
Shengcheng Luo, Quanquan Peng, Jun Lv +4
Employing a teleoperation system for gathering demonstrations offers the potential for more efficient learning of robot manipulation. However, teleoperating a robot arm equipped wi…
Low-Rank Similarity Mining for Multimodal Dataset Distillation
Yue Xu, Zhilin Lin, Yusong Qiu +2
Though dataset distillation has witnessed rapid development in recent years, the distillation of multimodal data, e.g., image-text pairs, poses unique and under-explored challenges…
HumanVLA: Towards Vision-Language Directed Object Rearrangement by Physical Humanoid
Xinyu Xu, Yizheng Zhang, Yong-Lu Li +2
Physical Human-Scene Interaction (HSI) plays a crucial role in numerous applications. However, existing HSI techniques are limited to specific object dynamics and privileged inform…
Primitive-based 3D Human-Object Interaction Modelling and Programming
Siqi Liu, Yong-Lu Li, Zhou Fang +3
Embedding Human and Articulated Object Interaction (HAOI) in 3D is an important direction for a deeper human activity understanding. Different from previous works that use parametr…
Revisit Human-Scene Interaction via Space Occupancy
Xinpeng Liu, Haowen Hou, Yanchao Yang +2
Human-scene Interaction (HSI) generation is a challenging task and crucial for various downstream tasks. However, one of the major obstacles is its limited data scale. High-quality…
Dancing with Still Images: Video Distillation via Static-Dynamic Disentanglement
Ziyu Wang, Yue Xu, Cewu Lu +1
Recently, dataset distillation has paved the way towards efficient machine learning, especially for image datasets. However, the distillation for videos, characterized by an exclus…
Symbol-LLM: Leverage Language Models for Symbolic System in Visual Human Activity Reasoning
Xiaoqian Wu, Yong-Lu Li, Jianhua Sun +1
Human reasoning can be understood as a cooperation between the intuitive, associative "System-1" and the deliberative, logical "System-2". For existing System-1-like methods in vis…
Bridging the Gap between Human Motion and Action Semantics via Kinematic Phrases
Xinpeng Liu, Yong-Lu Li, Ailing Zeng +3
Motion understanding aims to establish a reliable mapping between motion and action semantics, while it is a challenging many-to-many problem. An abstract action semantic (i.e., wa…
EgoPCA: A New Framework for Egocentric Hand-Object Interaction Understanding
Yue Xu, Yong-Lu Li, Zhemin Huang +4
With the surge in attention to Egocentric Hand-Object Interaction (Ego-HOI), large-scale datasets such as Ego4D and EPIC-KITCHENS have been proposed. However, most current research…
Distill Gold from Massive Ores: Bi-level Data Pruning towards Efficient Dataset Distillation
Yue Xu, Yong-Lu Li, Kaitong Cui +4
Data-efficient learning has garnered significant attention, especially given the current trend of large multi-modal models. Recently, dataset distillation has become an effective a…