331 citations · 844 across the 47 of their papers we have counts for
25 papers · 1 filter
Past, Future, All at Once: Mitigating Stability-Plasticity Dilemma via Post-hoc JANUS Rectification
Zhilong Zheng, Letian Tao, Yang Guan +7
Fine-tuning foundation models on new tasks inevitably suffer from catastrophic forgetting. While existing works attempt to mitigate this on the basis of parameter-efficient fine-tu…
Beyond Token-Local Imitation: Reward-Compatible Temporal Credit Assignment for On-Policy Distillation
Shiqi Liu, Zeyu He, Letian Tao +9
On-policy distillation (OPD) has emerged as an effective approach for large language model post-training, yet existing objectives face a trade-off between objective fidelity and op…
Mind Your Entropy: From Maximum Entropy to Trajectory Entropy-Constrained RL
Guojian Zhan, Likun Wang, Pengcheng Wang +4
Maximum entropy has become a mainstream off-policy reinforcement learning (RL) framework for balancing exploitation and exploration. However, two bottlenecks still limit further pe…
Distributional Soft Actor-Critic with Diffusion Policy
Tong Liu, Yinuo Wang, Xujie Song +6
Reinforcement learning has been proven to be highly effective in handling complex control tasks. Traditional methods typically use unimodal distributions, such as Gaussian distribu…
Global Optimality of Single-Timescale Actor-Critic under Continuous State-Action Space: A Study on Linear Quadratic Regulator
Xuyang Chen, Jingliang Duan, Lin Zhao
Actor-critic methods have achieved state-of-the-art performance in various challenging tasks. However, theoretical understandings of their performance remain elusive and challengin…
Enhanced DACER Algorithm with High Diffusion Efficiency
Yinuo Wang, Likun Wang, Mining Tan +11
Due to their expressive capacity, diffusion models have shown great promise in offline RL and imitation learning. Diffusion Actor-Critic with Entropy Regulator (DACER) extended thi…
Predictive Lagrangian Optimization for Constrained Reinforcement Learning
Tianqi Zhang, Puzhen Yuan, Guojian Zhan +6
Constrained optimization is popularly seen in reinforcement learning for addressing complex control tasks. From the perspective of dynamic system, iteratively solving a constrained…
Conformal Symplectic Optimization for Stable Reinforcement Learning
Yao Lyu, Xiangteng Zhang, Shengbo Eben Li +5
Training deep reinforcement learning (RL) agents necessitates overcoming the highly unstable nonconvex stochastic optimization inherent in the trial-and-error mechanism. To tackle…
Diffusion Actor-Critic with Entropy Regulator
Yinuo Wang, Likun Wang, Yuxuan Jiang +8
Reinforcement learning (RL) has proven highly effective in addressing complex decision-making and control tasks. However, in most traditional RL algorithms, the policy is typically…
Policy Bifurcation in Safe Reinforcement Learning
Wenjun Zou, Yao Lyu, Jie Li +7
Safe reinforcement learning (RL) offers advanced solutions to constrained optimal control problems. Existing studies in safe RL implicitly assume continuity in policy functions, wh…
Training Multi-layer Neural Networks on Ising Machine
Xujie Song, Tong Liu, Shengbo Eben Li +3
As a dedicated quantum device, Ising machines could solve large-scale binary optimization problems in milliseconds. There is emerging interest in utilizing Ising machines to train…
Distributional Soft Actor-Critic with Three Refinements
Jingliang Duan, Wenxuan Wang, Liming Xiao +6
Reinforcement learning (RL) has shown remarkable success in solving complex decision-making and control tasks. However, many model-free RL algorithms experience performance degrada…
Smoothing Policy Iteration for Zero-sum Markov Games
Yangang Ren, Yao Lyu, Wenxuan Wang +3
Zero-sum Markov Games (MGs) has been an efficient framework for multi-agent systems and robust control, wherein a minimax problem is constructed to solve the equilibrium policies.…
Enhance Sample Efficiency and Robustness of End-to-end Urban Autonomous Driving via Semantic Masked World Model
Zeyu Gao, Yao Mu, Chen Chen +4
End-to-end autonomous driving provides a feasible way to automatically maximize overall driving system performance by directly mapping the raw pixels from a front-facing camera to…
On the Optimization Landscape of Dynamic Output Feedback: A Case Study for Linear Quadratic Regulator
Jingliang Duan, Wenhan Cao, Yang Zheng +1
The convergence of policy gradient algorithms in reinforcement learning hinges on the optimization landscape of the underlying optimal control problem. Theoretical insights into th…
Global Convergence of Two-timescale Actor-Critic for Solving Linear Quadratic Regulator
Xuyang Chen, Jingliang Duan, Yingbin Liang +1
The actor-critic (AC) reinforcement learning algorithms have been the powerhouse behind many challenging applications. Nevertheless, its convergence is fragile in general. To study…
Self-learned Intelligence for Integrated Decision and Control of Automated Vehicles at Signalized Intersections
Yangang Ren, Jianhua Jiang, Dongjie Yu +4
Intersection is one of the most complex and accident-prone urban scenarios for autonomous driving wherein making safe and computationally efficient decisions is non-trivial. Curren…
Model-based Chance-Constrained Reinforcement Learning via Separated Proportional-Integral Lagrangian
Baiyu Peng, Jingliang Duan, Jianyu Chen +6
Safety is essential for reinforcement learning (RL) applied in the real world. Adding chance constraints (or probabilistic constraints) is a suitable way to enhance RL safety under…
Integrated Decision and Control: Towards Interpretable and Computationally Efficient Driving Intelligence
Yang Guan, Yangang Ren, Qi Sun +5
Decision and control are core functionalities of high-level automated vehicles. Current mainstream methods, such as functionality decomposition and end-to-end reinforcement learnin…
Separated Proportional-Integral Lagrangian for Chance Constrained Reinforcement Learning
Baiyu Peng, Yao Mu, Jingliang Duan +3
Safety is essential for reinforcement learning (RL) applied in real-world tasks like autonomous driving. Chance constraints which guarantee the satisfaction of state constraints at…
Mixed Policy Gradient: off-policy reinforcement learning driven jointly by data and model
Yang Guan, Jingliang Duan, Shengbo Eben Li +3
Reinforcement learning (RL) shows great potential in sequential decision-making. At present, mainstream RL algorithms are data-driven, which usually yield better asymptotic perform…
Safe Reinforcement Learning for Autonomous Vehicles through Parallel Constrained Policy Optimization
Lu Wen, Jingliang Duan, Shengbo Eben Li +2
Reinforcement learning (RL) is attracting increasing interests in autonomous driving due to its potential to solve complex classification and control problems. However, existing RL…
Improving Generalization of Reinforcement Learning with Minimax Distributional Soft Actor-Critic
Yangang Ren, Jingliang Duan, Shengbo Eben Li +2
Reinforcement learning (RL) has achieved remarkable performance in numerous sequential decision making and control tasks. However, a common problem is that learned nearly optimal p…
Distributional Soft Actor-Critic: Off-Policy Reinforcement Learning for Addressing Value Estimation Errors
Jingliang Duan, Yang Guan, Shengbo Eben Li +2
In reinforcement learning (RL), function approximation errors are known to easily lead to the Q-value overestimations, thus greatly reducing policy performance. This paper presents…