331 citations · 844 across the 47 of their papers we have counts for
49 papers
Past, Future, All at Once: Mitigating Stability-Plasticity Dilemma via Post-hoc JANUS Rectification
Zhilong Zheng, Letian Tao, Yang Guan +7
Fine-tuning foundation models on new tasks inevitably suffer from catastrophic forgetting. While existing works attempt to mitigate this on the basis of parameter-efficient fine-tu…
Beyond Token-Local Imitation: Reward-Compatible Temporal Credit Assignment for On-Policy Distillation
Shiqi Liu, Zeyu He, Letian Tao +9
On-policy distillation (OPD) has emerged as an effective approach for large language model post-training, yet existing objectives face a trade-off between objective fidelity and op…
FastDSAC: Enhancing Policy Plasticity via Constrained Exploration for Scalable Humanoid Locomotion
Guanchen Lu, Yajuan Dun, Yi Zhou +4
Scalable reinforcement learning has popularized high-throughput sampling architectures, which significantly compresses the training time for off-policy methods in robotic locomotio…
On the Optimization Landscape of Observer-based Dynamic Linear Quadratic Control
Jingliang Duan, Jie Li, Yinsong Ma +5
Understanding the optimization landscape of linear quadratic regulation (LQR) problems is fundamental to the design of efficient reinforcement learning solutions. Recent work has m…
STAPO: Stabilizing Reinforcement Learning for LLMs by Silencing Rare Spurious Tokens
Shiqi Liu, Zeyu He, Guojian Zhan +10
Reinforcement Learning (RL) has significantly improved large language model reasoning, but existing RL fine-tuning methods rely heavily on heuristic techniques such as entropy regu…
Off-policy Reinforcement Learning with Model-based Exploration Augmentation
Likun Wang, Xiangteng Zhang, Yinuo Wang +5
Exploration is fundamental to reinforcement learning (RL), as it determines how effectively an agent discovers and exploits the underlying structure of its environment to achieve o…
Mind Your Entropy: From Maximum Entropy to Trajectory Entropy-Constrained RL
Guojian Zhan, Likun Wang, Pengcheng Wang +4
Maximum entropy has become a mainstream off-policy reinforcement learning (RL) framework for balancing exploitation and exploration. However, two bottlenecks still limit further pe…
Distributional Soft Actor-Critic with Diffusion Policy
Tong Liu, Yinuo Wang, Xujie Song +6
Reinforcement learning has been proven to be highly effective in handling complex control tasks. Traditional methods typically use unimodal distributions, such as Gaussian distribu…
Distributional Soft Actor-Critic with Harmonic Gradient for Safe and Efficient Autonomous Driving in Multi-lane Scenarios
Feihong Zhang, Guojian Zhan, Bin Shuai +3
Reinforcement learning (RL), known for its self-evolution capability, offers a promising approach to training high-level autonomous driving systems. However, handling constraints r…
Global Optimality of Single-Timescale Actor-Critic under Continuous State-Action Space: A Study on Linear Quadratic Regulator
Xuyang Chen, Jingliang Duan, Lin Zhao
Actor-critic methods have achieved state-of-the-art performance in various challenging tasks. However, theoretical understandings of their performance remain elusive and challengin…
Enhanced DACER Algorithm with High Diffusion Efficiency
Yinuo Wang, Likun Wang, Mining Tan +11
Due to their expressive capacity, diffusion models have shown great promise in offline RL and imitation learning. Diffusion Actor-Critic with Entropy Regulator (DACER) extended thi…
Predictive Lagrangian Optimization for Constrained Reinforcement Learning
Tianqi Zhang, Puzhen Yuan, Guojian Zhan +6
Constrained optimization is popularly seen in reinforcement learning for addressing complex control tasks. From the perspective of dynamic system, iteratively solving a constrained…
Conformal Symplectic Optimization for Stable Reinforcement Learning
Yao Lyu, Xiangteng Zhang, Shengbo Eben Li +5
Training deep reinforcement learning (RL) agents necessitates overcoming the highly unstable nonconvex stochastic optimization inherent in the trial-and-error mechanism. To tackle…
Guiding Reinforcement Learning with Incomplete System Dynamics
Shuyuan Wang, Jingliang Duan, Nathan P. Lawrence +4
Model-free reinforcement learning (RL) is inherently a reactive method, operating under the assumption that it starts with no prior knowledge of the system and entirely depends on…
Diffusion Actor-Critic with Entropy Regulator
Yinuo Wang, Likun Wang, Yuxuan Jiang +8
Reinforcement learning (RL) has proven highly effective in addressing complex decision-making and control tasks. However, in most traditional RL algorithms, the policy is typically…
Policy Bifurcation in Safe Reinforcement Learning
Wenjun Zou, Yao Lyu, Jie Li +7
Safe reinforcement learning (RL) offers advanced solutions to constrained optimal control problems. Existing studies in safe RL implicitly assume continuity in policy functions, wh…
Integrated Drill Boom Hole-Seeking Control via Reinforcement Learning
Haoqi Yan, Haoyuan Xu, Hongbo Gao +3
Intelligent drill boom hole-seeking is a promising technology for enhancing drilling efficiency, mitigating potential safety hazards, and relieving human operators. Most existing i…
Training Multi-layer Neural Networks on Ising Machine
Xujie Song, Tong Liu, Shengbo Eben Li +3
As a dedicated quantum device, Ising machines could solve large-scale binary optimization problems in milliseconds. There is emerging interest in utilizing Ising machines to train…
Optimization Landscape of Policy Gradient Methods for Discrete-time Static Output Feedback
Jingliang Duan, Jie Li, Xuyang Chen +3
In recent times, significant advancements have been made in delving into the optimization landscape of policy gradient methods for achieving optimal control in linear time-invarian…
Distributional Soft Actor-Critic with Three Refinements
Jingliang Duan, Wenxuan Wang, Liming Xiao +6
Reinforcement learning (RL) has shown remarkable success in solving complex decision-making and control tasks. However, many model-free RL algorithms experience performance degrada…
Smoothing Policy Iteration for Zero-sum Markov Games
Yangang Ren, Yao Lyu, Wenxuan Wang +3
Zero-sum Markov Games (MGs) has been an efficient framework for multi-agent systems and robust control, wherein a minimax problem is constructed to solve the equilibrium policies.…
Safe Model-Based Reinforcement Learning with an Uncertainty-Aware Reachability Certificate
Dongjie Yu, Wenjun Zou, Yujie Yang +4
Safe reinforcement learning (RL) that solves constraint-satisfactory policies provides a promising way to the broader safety-critical applications of RL in real-world problems such…
Enhance Sample Efficiency and Robustness of End-to-end Urban Autonomous Driving via Semantic Masked World Model
Zeyu Gao, Yao Mu, Chen Chen +4
End-to-end autonomous driving provides a feasible way to automatically maximize overall driving system performance by directly mapping the raw pixels from a front-facing camera to…
On the Optimization Landscape of Dynamic Output Feedback: A Case Study for Linear Quadratic Regulator
Jingliang Duan, Wenhan Cao, Yang Zheng +1
The convergence of policy gradient algorithms in reinforcement learning hinges on the optimization landscape of the underlying optimal control problem. Theoretical insights into th…