Policy Gradients using Variational Quantum Circuits

Abstract

Variational Quantum Circuits are being used as versatile Quantum Machine Learning models. Some empirical results exhibit an advantage in supervised and generative learning tasks. However, when applied to Reinforcement Learning, less is known. In this work, we considered a Variational Quantum Circuit composed of a low-depth hardware-efficient ansatz as the parameterized policy of a Reinforcement Learning agent. We show that an $\epsilon$-approximation of the policy gradient can be obtained using a logarithmic number of samples concerning the total number of parameters. We empirically verify that such quantum models behave similarly or even outperform typical classical neural networks used in standard benchmarking environments and in quantum control, using only a fraction of the parameters. Moreover, we study the Barren Plateau phenomenon in quantum policy gradients using the Fisher Information Matrix spectrum.

References in corpus (20)

• Quantum Computing in the NISQ era and beyond
• Variational Quantum Algorithms
• Noisy intermediate-scale quantum (NISQ) algorithms
• Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model
• Evaluating analytic gradients on quantum hardware
• Expressibility and entangling capability of parameterized quantum circuits for hybrid quantum-classical algorithms
• The effect of data encoding on the expressive power of variational quantum machine learning models
• Power of data in quantum machine learning
• Quantum Natural Gradient
• Quantum-enhanced machine learning
• Robust data encodings for quantum classifiers
• Stochastic gradient descent for hybrid quantum-classical optimization
• Quantum speedup for active learning agents
• Experimental quantum speed-up in reinforcement learning agents
• When does reinforcement learning stand out in quantum control? A comparative study on state preparation
• Fisher Information in Noisy Intermediate-Scale Quantum Applications
• Advances in Quantum Reinforcement Learning
• Escaping from the Barren Plateau via Gaussian Initializations in Deep Variational Quantum Circuits
• Short Quantum Circuits in Reinforcement Learning Policies for the Vehicle Routing Problem
• Comprehensive Review of Deep Reinforcement Learning Methods and Applications in Economics