Variational Quantum Reinforcement Learning via Evolutionary Optimization

Abstract

Recent advance in classical reinforcement learning (RL) and quantum computation (QC) points to a promising direction of performing RL on a quantum computer. However, potential applications in quantum RL are limited by the number of qubits available in the modern quantum devices. Here we present two frameworks of deep quantum RL tasks using a gradient-free evolution optimization: First, we apply the amplitude encoding scheme to the Cart-Pole problem; Second, we propose a hybrid framework where the quantum RL agents are equipped with hybrid tensor network-variational quantum circuit (TN-VQC) architecture to handle inputs with dimensions exceeding the number of qubits. This allows us to perform quantum RL on the MiniGrid environment with 147-dimensional inputs. We demonstrate the quantum advantage of parameter saving using the amplitude encoding. The hybrid TN-VQC architecture provides a natural way to perform efficient compression of the input dimension, enabling further quantum RL applications on noisy intermediate-scale quantum devices.

References in corpus (26)

• Supplementary information for "Quantum supremacy using a programmable superconducting processor"
• The density-matrix renormalization group in the age of matrix product states
• Matrix Product States, Projected Entangled Pair States, and variational renormalization group methods for quantum spin systems
• A Quantum Approximate Optimization Algorithm
• DMRG and periodic boundary conditions: a quantum information perspective
• Quantum reinforcement learning
• Quantum convolutional neural network for classical data classification
• Quantum agents in the Gym: a variational quantum algorithm for deep Q-learning
• Model-Free Quantum Control with Reinforcement Learning
• Tree Tensor Networks for Generative Modeling
• Tensor Networks in a Nutshell
• Classification with Quantum Machine Learning: A Survey
• TensorNetwork for Machine Learning
• Generalized quantum circuit differentiation rules
• Natural Evolutionary Strategies for Variational Quantum Computation
• Quantum Architecture Search via Deep Reinforcement Learning
• Hybrid Quantum-Classical Graph Convolutional Network
• Recurrent Quantum Neural Networks
• Quantum Quantile Mechanics: Solving Stochastic Differential Equations for Generating Time-Series
• Quantum Long Short-Term Memory
• QuClassi: A Hybrid Deep Neural Network Architecture based on Quantum State Fidelity
• Variational Quanvolutional Neural Networks with enhanced image encoding
• QTN-VQC: An End-to-End Learning framework for Quantum Neural Networks
• Quantum Convolutional Neural Networks for High Energy Physics Data Analysis
• Introduction to Quantum Reinforcement Learning: Theory and PennyLane-based Implementation
• Towards Building A Facial Identification System Using Quantum Machine Learning Techniques

Cited by in corpus (11)

• Quantum computing for finance
• Self-Correcting Quantum Many-Body Control using Reinforcement Learning with Tensor Networks
• Tensor networks for quantum machine learning
• Quantum Architecture Search with Meta-learning
• Quantum Deep Reinforcement Learning for Robot Navigation Tasks
• Optimizing quantum noise-induced reservoir computing for nonlinear and chaotic time series prediction
• Using Differential Evolution to avoid local minima in Variational Quantum Algorithms
• Hybrid actor-critic algorithm for quantum reinforcement learning at CERN beam lines
• Transfer Learning Analysis of Variational Quantum Circuits
• The Cytnx Library for Tensor Networks
• Quantum-Classical Autoencoder Architectures for End-to-End Radio Communication