19 citations · 41 across the 20 of their papers we have counts for
22 papers
From Video to Vision: Learning Generalizable Humanoid Carrying from Human Motion
Ziluo Ding
Most existing approaches rely on video or motion-capture data for reinforcement learning (RL) training; however, all of these methods require an upper-level module to generate refe…
HAF: Adapting Generalist VLAs to Humanoid Whole-Body Loco-manipulation via Hierarchical Action Flow and Spectral Latent RL
Langzhe Gu, Chengkai Hou, Meng Li +14
Humanoid robots hold great promise as general-purpose agents in human-centered environments, yet generalist vision-language-action (VLA) foundation models are not readily applicabl…
In-Context Reinforcement Learning under Non-Stationarity: A Survey
A Run, Ziluo Ding
The development of decision-pretrained transformers, algorithm distillation, long-context meta-RL, and retrieval-augmented agents has renewed interest in in-context reinforcement l…
HEX: Humanoid-Aligned Experts for Cross-Embodiment Whole-Body Manipulation
Shuanghao Bai, Meng Li, Xinyuan Lv +15
Humans achieve complex manipulation through coordinated whole-body control, whereas most Vision-Language-Action (VLA) models treat robot body parts largely independently, making hi…
General Humanoid Whole-Body Control via Pretraining and Fast Adaptation
Zepeng Wang, Jiangxing Wang, Shiqing Yao +8
Learning a general whole-body controller for humanoid robots remains challenging due to the diversity of motion distributions, the difficulty of fast adaptation, and the need for r…
SENTINEL: A Fully End-to-End Language-Action Model for Humanoid Whole Body Control
Yuxuan Wang, Haobin Jiang, Shiqing Yao +2
Existing humanoid control systems often rely on teleoperation or modular generation pipelines that separate language understanding from physical execution. However, the former is e…
Seeing the Unseen in Low-light Spike Streams
Liwen Hu, Yang Li, Mianzhi Liu +5
Spike camera, a type of neuromorphic sensor with high-temporal resolution, shows great promise for high-speed visual tasks. Unlike traditional cameras, spike camera continuously ac…
JAEGER: Dual-Level Humanoid Whole-Body Controller
Ziluo Ding, Haobin Jiang, Yuxuan Wang +7
This paper presents JAEGER, a dual-level whole-body controller for humanoid robots that addresses the challenges of training a more robust and versatile policy. Unlike traditional…
RL from Physical Feedback: Aligning Large Motion Models with Humanoid Control
Junpeng Yue, Zepeng Wang, Yuxuan Wang +7
This paper focuses on a critical challenge in robotics: translating text-driven human motions into executable actions for humanoid robots, enabling efficient and cost-effective lea…
From Experts to a Generalist: Toward General Whole-Body Control for Humanoid Robots
Yuxuan Wang, Ming Yang, Ziluo Ding +5
Achieving general agile whole-body control on humanoid robots remains a major challenge due to diverse motion demands and data conflicts. While existing frameworks excel in trainin…
SELU: Self-Learning Embodied MLLMs in Unknown Environments
Boyu Li, Haobin Jiang, Ziluo Ding +4
Recently, multimodal large language models (MLLMs) have demonstrated strong visual understanding and decision-making capabilities, enabling the exploration of autonomously improvin…
Egocentric Vision Language Planning
Zhirui Fang, Ming Yang, Weishuai Zeng +5
We explore leveraging large multi-modal models (LMMs) and text2image models to build a more general embodied agent. LMMs excel in planning long-horizon tasks over symbolic abstract…
A Survey on Game Playing Agents and Large Models: Methods, Applications, and Challenges
Xinrun Xu, Yuxin Wang, Chaoyi Xu +4
The swift evolution of Large-scale Models (LMs), either language-focused or multi-modal, has garnered extensive attention in both academy and industry. But despite the surge in int…
Cradle: Empowering Foundation Agents Towards General Computer Control
Weihao Tan, Wentao Zhang, Xinrun Xu +25
Despite the success in specific scenarios, existing foundation agents still struggle to generalize across various virtual scenarios, mainly due to the dramatically different encaps…
Settling Decentralized Multi-Agent Coordinated Exploration by Novelty Sharing
Haobin Jiang, Ziluo Ding, Zongqing Lu
Exploration in decentralized cooperative multi-agent reinforcement learning faces two challenges. One is that the novelty of global states is unavailable, while the novelty of loca…
Learning to Robustly Reconstruct Low-light Dynamic Scenes from Spike Streams
Liwen Hu, Ziluo Ding, Mianzhi Liu +2
As a neuromorphic sensor with high temporal resolution, spike camera can generate continuous binary spike streams to capture per-pixel light intensity. We can use reconstruction me…
Unsupervised Optical Flow Estimation with Dynamic Timing Representation for Spike Camera
Lujie Xia, Ziluo Ding, Rui Zhao +5
Efficiently selecting an appropriate spike stream data length to extract precise information is the key to the spike vision tasks. To address this issue, we propose a dynamic timin…
Reinforcement Learning Friendly Vision-Language Model for Minecraft
Haobin Jiang, Junpeng Yue, Hao Luo +2
One of the essential missions in the AI research community is to build an autonomous embodied agent that can achieve high-level performance across a wide spectrum of tasks. However…
Entity Divider with Language Grounding in Multi-Agent Reinforcement Learning
Ziluo Ding, Wanpeng Zhang, Junpeng Yue +3
We investigate the use of natural language to drive the generalization of policies in multi-agent settings. Unlike single-agent settings, the generalization of policies should also…
Optical Flow Estimation for Spiking Camera
Liwen Hu, Rui Zhao, Ziluo Ding +4
As a bio-inspired sensor with high temporal resolution, the spiking camera has an enormous potential in real applications, especially for motion estimation in high-speed scenes. Ho…
Spatio-Temporal Recurrent Networks for Event-Based Optical Flow Estimation
Ziluo Ding, Rui Zhao, Jiyuan Zhang +4
Event camera has offered promising alternative for visual perception, especially in high speed and high dynamic range scenes. Recently, many deep learning methods have shown great…
Learning Individually Inferred Communication for Multi-Agent Cooperation
Ziluo Ding, Tiejun Huang, Zongqing Lu
Communication lays the foundation for human cooperation. It is also crucial for multi-agent cooperation. However, existing work focuses on broadcast communication, which is not onl…