12 papers
A Single Diffusion-Policy Controller for Multi-Task Block Pushing with Zero-Shot Sim-to-Real Transfer
Haitong Ma, Haldun Balim, Yang Hu +2
Diffusion policies have shown promising empirical performance in representing and learning complex maneuvers for robots using behavior cloning (BC). In this paper, we explore train…
GeMPO: Generalized Measure Matching for Online Diffusion Reinforcement Learning
Haitong Ma, Chenxiao Gao, Tianyi Chen +2
A commonly used family of RL algorithms for diffusion policies conducts softmax reweighting over samples from the behavior policy, which often induces an overgreedy policy and fail…
Spectral Ghost in Representation Learning: from Component Analysis to Self-Supervised Learning
Bo Dai, Na Li, Dale Schuurmans
Self-supervised learning (SSL) has improved empirical performance by unleashing the power of unlabeled data for practical applications. Specifically, SSL extracts the representatio…
Max-Entropy Reinforcement Learning with Flow Matching and A Case Study on LQR
Yuyang Zhang, Yang Hu, Bo Dai +1
Soft actor-critic (SAC) is a popular algorithm for max-entropy reinforcement learning. In practice, the energy-based policies in SAC are often approximated using simple policy clas…
Offline Imitation Learning upon Arbitrary Demonstrations by Pre-Training Dynamics Representations
Haitong Ma, Bo Dai, Zhaolin Ren +2
Limited data has become a major bottleneck in scaling up offline imitation learning (IL). In this paper, we propose enhancing IL performance under limited expert data by introducin…
One-Step Flow Policy Mirror Descent
Tianyi Chen, Haitong Ma, Na Li +2
Diffusion policies have achieved great success in online reinforcement learning (RL) due to their strong expressive capacity. However, the inference of diffusion policy models reli…
Efficient Online Reinforcement Learning for Diffusion Policy
Haitong Ma, Tianyi Chen, Kai Wang +2
Diffusion policies have achieved superior performance in imitation learning and offline reinforcement learning (RL) due to their rich expressiveness. However, the conventional diff…
Scalable spectral representations for multi-agent reinforcement learning in network MDPs
Zhaolin Ren, Runyu Zhang, Bo Dai +1
Network Markov Decision Processes (MDPs), a popular model for multi-agent control, pose a significant challenge to efficient learning due to the exponential growth of the global st…
Primal-Dual Spectral Representation for Off-policy Evaluation
Yang Hu, Tianyi Chen, Na Li +2
Off-policy evaluation (OPE) is one of the most fundamental problems in reinforcement learning (RL) to estimate the expected long-term payoff of a given target policy with only expe…
Efficient Duple Perturbation Robustness in Low-rank MDPs
Yang Hu, Haitong Ma, Bo Dai +1
The pursuit of robustness has recently been a popular topic in reinforcement learning (RL) research, yet the existing methods generally suffer from efficiency issues that obstruct…
Skill Transfer and Discovery for Sim-to-Real Learning: A Representation-Based Viewpoint
Haitong Ma, Zhaolin Ren, Bo Dai +1
We study sim-to-real skill transfer and discovery in the context of robotics control using representation learning. We draw inspiration from spectral decomposition of Markov decisi…
Stochastic Nonlinear Control via Finite-dimensional Spectral Dynamic Embedding
Zhaolin Ren, Tongzheng Ren, Haitong Ma +2
This paper proposes an approach, Spectral Dynamics Embedding Control (SDEC), to optimal control for nonlinear stochastic systems. This method reveals an infinite-dimensional featur…