13 citations · 27 across the 27 of their papers we have counts for
27 papers
Hydra-0: Action Flow for Generalist World Modeling and Control
Hongyu Li, Bowen Wen, Xinghao Zhu +8
We introduce Hydra-0, a generalist world model conditioned on action flow, which represents robot actions as pixel motion. This shared visual interface enables generalist world mod…
Learning Dexterous Manipulation Using Contact Wrench Guidance From Human Demonstration
Xinghao Zhu, Zixi Liu, Shalin Jain +18
Dexterous robot manipulation can benefit from the abundance of human demonstrations, but transferring such demonstrations to robot policies remains challenging. We present Contact…
REAR: Test-time Preference Realignment through Reward Decomposition
Fuxiang Zhang, Pengcheng Wang, Chenran Li +6
Aligning large language models (LLMs) with diverse user preferences is a critical yet challenging task. While post-training methods can adapt models to specific needs, they often r…
TEXEDO : Test Time Scaling for Controller-aware Language-conditioned Humanoid Motion Generation
Jianuo Cao, Yuxin Chen, Yuzhen Song +3
Text-conditioned motion generation is a promising interface for programming humanoid robots, yet current generators are often trained on human motion datasets retargeted to robot m…
CoorDex: Coordinating Body and Hand Priors for Continuous Dexterous Humanoid Loco-Manipulation
Sikai Li, Shuning Li, Zhenyu Wei +3
Humanoid loco-manipulation is often simplified into a stop-and-go process: walking to an object, stopping to manipulate it, and then resuming locomotion. It also commonly relies on…
MotionBricks: Scalable Real-Time Motions with Modular Latent Generative Model and Smart Primitives
Tingwu Wang, Olivier Dionne, Michael De Ruyter +13
Despite transformative advances in generative motion synthesis, real-time interactive motion control remains dominated by traditional techniques. In this work, we identify two key…
CHIP: Adaptive Compliance for Humanoid Control through Hindsight Perturbation
Sirui Chen, Zi-ang Cao, Zhengyi Luo +7
Recent progress in humanoid robots has unlocked agile locomotion skills, including backflipping, running, and crawling. Yet it remains challenging for a humanoid robot to perform f…
Isaac Lab: A GPU-Accelerated Simulation Framework for Multi-Modal Robot Learning
NVIDIA, :, Mayank Mittal +104
We present Isaac Lab, the natural successor to Isaac Gym, which extends the paradigm of GPU-native robotics simulation into the era of large-scale multi-modal learning. Isaac Lab c…
SONIC: Supersizing Motion Tracking for Natural Humanoid Whole-Body Control
Zhengyi Luo, Ye Yuan, Tingwu Wang +25
Despite the rise of billion-parameter foundation models trained across thousands of graphical processing units (GPUs), similar scaling gains have not been shown for humanoid contro…
DexCtrl: Towards Sim-to-Real Dexterity with Adaptive Controller Learning
Shuqi Zhao, Ke Yang, Yuxin Chen +5
Dexterous manipulation has seen remarkable progress in recent years, with policies capable of executing many complex and contact-rich tasks in simulation. However, transferring the…
Residual Policy Gradient: A Reward View of KL-regularized Objective
Pengcheng Wang, Xinghao Zhu, Yuxin Chen +3
Reinforcement Learning and Imitation Learning have achieved widespread success in many domains but remain constrained during real-world deployment. One of the main issues is the ad…
COMPASS: Cross-embodiment Mobility Policy via Residual RL and Skill Synthesis
Wei Liu, Huihua Zhao, Chenran Li +4
As robots are increasingly deployed in diverse application domains, enabling robust mobility across different embodiments has become a critical challenge. Classical mobility stacks…
Optimal Driver Warning Generation in Dynamic Driving Environment
Chenran Li, Aolin Xu, Enna Sachdeva +2
The driver warning system that alerts the human driver about potential risks during driving is a key feature of an advanced driver assistance system. Existing driver warning techno…
DexH2R: Task-oriented Dexterous Manipulation from Human to Robots
Shuqi Zhao, Xinghao Zhu, Yuxin Chen +5
Dexterous manipulation is a critical aspect of human capability, enabling interaction with a wide variety of objects. Recent advancements in learning from human demonstrations and…
X-MOBILITY: End-To-End Generalizable Navigation via World Modeling
Wei Liu, Huihua Zhao, Chenran Li +5
General-purpose navigation in challenging environments remains a significant problem in robotics, with current state-of-the-art approaches facing myriad limitations. Classical appr…
WOMD-Reasoning: A Large-Scale Dataset for Interaction Reasoning in Driving
Yiheng Li, Cunxin Fan, Chongjian Ge +9
Language models uncover unprecedented abilities in analyzing driving scenarios, owing to their limitless knowledge accumulated from text-based pre-training. Naturally, they should…
Residual-MPPI: Online Policy Customization for Continuous Control
Pengcheng Wang, Chenran Li, Catherine Weaver +4
Policies developed through Reinforcement Learning (RL) and Imitation Learning (IL) have shown great potential in continuous control tasks, but real-world applications often require…
MEReQ: Max-Ent Residual-Q Inverse RL for Sample-Efficient Alignment from Intervention
Yuxin Chen, Chen Tang, Jianglan Wei +6
Aligning robot behavior with human preferences is crucial for deploying embodied AI agents in human-centered environments. A promising solution is interactive imitation learning fr…
Adaptive Energy Regularization for Autonomous Gait Transition and Energy-Efficient Quadruped Locomotion
Boyuan Liang, Lingfeng Sun, Xinghao Zhu +6
In reinforcement learning for legged robot locomotion, crafting effective reward strategies is crucial. Pre-defined gait patterns and complex reward systems are widely used to stab…
DrPlanner: Diagnosis and Repair of Motion Planners for Automated Vehicles Using Large Language Models
Yuanfei Lin, Chenran Li, Mingyu Ding +3
Motion planners are essential for the safe operation of automated vehicles across various scenarios. However, no motion planning algorithm has achieved perfection in the literature…
Quantifying Agent Interaction in Multi-agent Reinforcement Learning for Cost-efficient Generalization
Yuxin Chen, Chen Tang, Ran Tian +4
Generalization poses a significant challenge in Multi-agent Reinforcement Learning (MARL). The extent to which an agent is influenced by unseen co-players depends on the agent's po…
Pre-training on Synthetic Driving Data for Trajectory Prediction
Yiheng Li, Seth Z. Zhao, Chenfeng Xu +5
Accumulating substantial volumes of real-world driving data proves pivotal in the realm of trajectory forecasting for autonomous driving. Given the heavy reliance of current trajec…
Residual Q-Learning: Offline and Online Policy Customization without Value
Chenran Li, Chen Tang, Haruki Nishimura +3
Imitation Learning (IL) is a widely used framework for learning imitative behavior from demonstrations. It is especially appealing for solving complex real-world tasks where handcr…
Editing Driver Character: Socially-Controllable Behavior Generation for Interactive Traffic Simulation
Wei-Jer Chang, Chen Tang, Chenran Li +3
Traffic simulation plays a crucial role in evaluating and improving autonomous driving planning systems. After being deployed on public roads, autonomous vehicles need to interact…