A Convergence Result for Regularized Actor-Critic Methods
arXiv:1907.06138
Abstract
In this paper, we present a probability one convergence proof, under suitable conditions, of a certain class of actor-critic algorithms for finding approximate solutions to entropy-regularized MDPs using the machinery of stochastic approximation. To obtain this overall result, we prove the convergence of policy evaluation with general regularizers when using linear approximation architectures and show convergence of entropy-regularized policy improvement.
References in corpus (8)
- Trust Region Policy Optimization
- Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agents
- Bridging the Gap Between Value and Policy Based Reinforcement Learning
- Equivalence Between Policy Gradients and Soft Q-Learning
- SBEED: Convergent Reinforcement Learning with Nonlinear Function Approximation
- A Theory of Regularized Markov Decision Processes
- Combining policy gradient and Q-learning
- Actor-Critic Algorithms for Learning Nash Equilibria in N-player General-Sum Games