Quantum Reinforcement Learning via Policy Iteration

Abstract

Quantum computing has shown the potential to substantially speed up machine learning applications, in particular for supervised and unsupervised learning. Reinforcement learning, on the other hand, has become essential for solving many decision making problems and policy iteration methods remain the foundation of such approaches. In this paper, we provide a general framework for performing quantum reinforcement learning via policy iteration. We validate our framework by designing and analyzing: \emph{quantum policy evaluation} methods for infinite horizon discounted problems by building quantum states that approximately encode the value function of a policy $\pi$; and \emph{quantum policy improvement} methods by post-processing measurement outcomes on these quantum states. Last, we study the theoretical and experimental performance of our quantum algorithms on two environments from OpenAI's Gym.

References in corpus (9)

• Supplementary information for "Quantum supremacy using a programmable superconducting processor"
• Quantum Machine Learning
• Quantum principal component analysis
• Quantum generative adversarial learning
• Quantum singular value transformation and beyond: exponential improvements for quantum matrix arithmetics
• Quantum reinforcement learning
• Quantum agents in the Gym: a variational quantum algorithm for deep Q-learning
• Quantum gradient descent for linear systems and least squares
• Optimizing quantum optimization algorithms via faster quantum gradient computation

Cited by in corpus (8)

• Quantum Deep Hedging
• Quantum Deep Reinforcement Learning for Robot Navigation Tasks
• Quantum reinforcement learning in continuous action space
• Quantum framework for Reinforcement Learning: Integrating Markov decision process, quantum arithmetic, and trajectory search
• Fourier Analysis of Variational Quantum Circuits for Supervised Learning
• Comprehensive Library of Variational LSE Solvers
• Warm-Start Variational Quantum Policy Iteration
• Exploring fixed points and eigenstates of quantum systems with reinforcement learning