41 citations · 88 across the 20 of their papers we have counts for
20 papers
RxBrain: Embodied Cognition Foundation Model with Joint Language-Visual Reasoning and Imagination
Haotian Liang, Mingkang Chen, Yufei Huang +27
Embodied cognition requires agents to connect high-level task reasoning with the physical states to be achieved. We introduce Hy-Embodied-RxBrain, an embodied cognition foundation…
MetaDiffuser: Diffusion Model as Conditional Planner for Offline Meta-RL
Fei Ni, Jianye Hao, Yao Mu +4
Recently, diffusion model shines as a promising backbone for the sequence modeling paradigm in offline reinforcement learning(RL). However, these works mostly lack the generalizati…
EmbodiedGPT: Vision-Language Pre-Training via Embodied Chain of Thought
Yao Mu, Qinglong Zhang, Mengkang Hu +7
Embodied AI is a crucial frontier in robotics, capable of planning and executing action sequences for robots to accomplish long-horizon tasks in physical environments. In this work…
EC^2: Emergent Communication for Embodied Control
Yao Mu, Shunyu Yao, Mingyu Ding +2
Embodied control requires agents to leverage multi-modal pre-training to quickly learn how to act in new environments, where video demonstrations contain visual and motion details…
AdaptDiffuser: Diffusion Models as Adaptive Self-evolving Planners
Zhixuan Liang, Yao Mu, Mingyu Ding +3
Diffusion models have demonstrated their powerful generative capability in many tasks, with great potential to serve as a paradigm for offline reinforcement learning. However, the…
MaskPlace: Fast Chip Placement via Reinforced Visual Representation Learning
Yao Lai, Yao Mu, Ping Luo
Placement is an essential task in modern chip design, aiming at placing millions of circuit modules on a 2D chip canvas. Unlike the human-centric solution, which requires months of…
Prototypical context-aware dynamics generalization for high-dimensional model-based reinforcement learning
Junjie Wang, Yao Mu, Dong Li +6
The latent world model provides a promising way to learn policies in a compact latent space for tasks with high-dimensional observations, however, its generalization across diverse…
Decomposed Mutual Information Optimization for Generalized Context in Meta-Reinforcement Learning
Yao Mu, Yuzheng Zhuang, Fei Ni +4
Adapting to the changes in transition dynamics is essential in robotic applications. By learning a conditional policy with a compact context, context-aware meta-reinforcement learn…
Enhance Sample Efficiency and Robustness of End-to-end Urban Autonomous Driving via Semantic Masked World Model
Zeyu Gao, Yao Mu, Chen Chen +4
End-to-end autonomous driving provides a feasible way to automatically maximize overall driving system performance by directly mapping the raw pixels from a front-facing camera to…
EUCLID: Towards Efficient Unsupervised Reinforcement Learning with Multi-choice Dynamics Model
Yifu Yuan, Jianye Hao, Fei Ni +6
Unsupervised reinforcement learning (URL) poses a promising paradigm to learn useful behaviors in a task-agnostic environment without the guidance of extrinsic rewards to facilitat…
CtrlFormer: Learning Transferable State Representation for Visual Control via Transformer
Yao Mu, Shoufa Chen, Mingyu Ding +3
Transformer has achieved great successes in learning vision and language representation, which is general across various downstream tasks. In visual control, learning transferable…
CO^3: Cooperative Unsupervised 3D Representation Learning for Autonomous Driving
Runjian Chen, Yao Mu, Runsen Xu +5
Unsupervised contrastive learning for indoor-scene point clouds has achieved great successes. However, unsupervised learning point clouds in outdoor scenes remains challenging beca…
Flow-based Recurrent Belief State Learning for POMDPs
Xiaoyu Chen, Yao Mu, Ping Luo +2
Partially Observable Markov Decision Process (POMDP) provides a principled and generic framework to model real world sequential decision making processes but yet remains unsolved,…
Scale-Equivalent Distillation for Semi-Supervised Object Detection
Qiushan Guo, Yao Mu, Jianyu Chen +3
Recent Semi-Supervised Object Detection (SS-OD) methods are mainly based on self-training, i.e., generating hard pseudo-labels by a teacher model on unlabeled data as supervisory s…
Don't Touch What Matters: Task-Aware Lipschitz Data Augmentation for Visual Reinforcement Learning
Zhecheng Yuan, Guozheng Ma, Yao Mu +5
One of the key challenges in visual Reinforcement Learning (RL) is to learn policies that can generalize to unseen environments. Recently, data augmentation techniques aiming at en…
Model-based Chance-Constrained Reinforcement Learning via Separated Proportional-Integral Lagrangian
Baiyu Peng, Jingliang Duan, Jianyu Chen +6
Safety is essential for reinforcement learning (RL) applied in the real world. Adding chance constraints (or probabilistic constraints) is a suitable way to enhance RL safety under…
Separated Proportional-Integral Lagrangian for Chance Constrained Reinforcement Learning
Baiyu Peng, Yao Mu, Jingliang Duan +3
Safety is essential for reinforcement learning (RL) applied in real-world tasks like autonomous driving. Chance constraints which guarantee the satisfaction of state constraints at…
Steadily Learn to Drive with Virtual Memory
Yuhang Zhang, Yao Mu, Yujie Yang +4
Reinforcement learning has shown great potential in developing high-level autonomous driving. However, for high-dimensional tasks, current RL methods suffer from low data efficienc…
Model-Based Actor-Critic with Chance Constraint for Stochastic System
Baiyu Peng, Yao Mu, Yang Guan +3
Safety is essential for reinforcement learning (RL) applied in real-world situations. Chance constraints are suitable to represent the safety requirements in stochastic systems. Pr…
Mixed Reinforcement Learning with Additive Stochastic Uncertainty
Yao Mu, Shengbo Eben Li, Chang Liu +4
Reinforcement learning (RL) methods often rely on massive exploration data to search optimal policies, and suffer from poor sampling efficiency. This paper presents a mixed reinfor…