Variational Quantum Reinforcement Learning via Evolutionary Optimization

Abstract

Recent advance in classical reinforcement learning (RL) and quantum computation (QC) points to a promising direction of performing RL on a quantum computer. However, potential applications in quantum RL are limited by the number of qubits available in the modern quantum devices. Here we present two frameworks of deep quantum RL tasks using a gradient-free evolution optimization: First, we apply the amplitude encoding scheme to the Cart-Pole problem; Second, we propose a hybrid framework where the quantum RL agents are equipped with hybrid tensor network-variational quantum circuit (TN-VQC) architecture to handle inputs with dimensions exceeding the number of qubits. This allows us to perform quantum RL on the MiniGrid environment with 147-dimensional inputs. We demonstrate the quantum advantage of parameter saving using the amplitude encoding. The hybrid TN-VQC architecture provides a natural way to perform efficient compression of the input dimension, enabling further quantum RL applications on noisy intermediate-scale quantum devices.

References in corpus (76)

• Quantum Computing in the NISQ era and beyond
• Supplementary information for "Quantum supremacy using a programmable superconducting processor"
• Quantum Mechanics helps in searching for a needle in a haystack
• The density-matrix renormalization group in the age of matrix product states
• The density-matrix renormalization group
• Hardware-efficient Variational Quantum Eigensolver for Small Molecules and Quantum Magnets
• Supervised learning with quantum enhanced feature spaces
• The theory of variational hybrid quantum-classical algorithms
• Matrix Product States, Projected Entangled Pair States, and variational renormalization group methods for quantum spin systems
• A Quantum Approximate Optimization Algorithm
• Quantum Circuit Learning
• Quantum Convolutional Neural Networks
• Parameterized quantum circuits as machine learning models
• Evaluating analytic gradients on quantum hardware
• Circuit-centric quantum classifiers
• A Theory of Fault-Tolerant Quantum Computation
• Expressibility and entangling capability of parameterized quantum circuits for hybrid quantum-classical algorithms
• Quantum Computational Supremacy
• Stabilizer Codes and Quantum Error Correction
• OpenAI Gym
• DMRG and periodic boundary conditions: a quantum information perspective
• Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning
• Quantum generative adversarial networks
• Quantum Generative Adversarial Networks for Learning and Loading Random Distributions
• Transfer learning in hybrid classical-quantum neural networks
• Quantum reinforcement learning
• Quantum convolutional neural network for classical data classification
• Renormalization and tensor product states in spin chains and lattices
• Towards Quantum Machine Learning with Tensor Networks
• The Expressive Power of Parameterized Quantum Circuits
• Active learning machine learns to create new quantum experiments
• Unsupervised Generative Modeling Using Matrix Product States
• Reinforcement Learning with Neural Networks for Quantum Feedback
• Agent57: Outperforming the Atari Human Benchmark
• Quantum agents in the Gym: a variational quantum algorithm for deep Q-learning
• Entanglement in a quantum annealing processor
• Solving nonlinear differential equations with differentiable quantum circuits
• Optimizing Quantum Error Correction Codes with Reinforcement Learning
• Encoding of Matrix Product States into Quantum Circuits of One- and Two-Qubit Gates
• Model-Free Quantum Control with Reinforcement Learning
• Tree Tensor Networks for Generative Modeling
• Tensor Networks in a Nutshell
• Improving Exploration in Evolution Strategies for Deep Reinforcement Learning via a Population of Novelty-Seeking Agents
• Machine Learning by Unitary Tensor Network of Hierarchical Tree Structure
• Efficient Arbitrary Simultaneously Entangling Gates on a trapped-ion quantum computer
• Learning Relevant Features of Data with Multi-scale Tensor Networks
• Neural Belief-Propagation Decoders for Quantum Error-Correcting Codes
• Application of Quantum Machine Learning using the Quantum Variational Classifier Method to High Energy Physics Analysis at the LHC on IBM Quantum Computer Simulator and Hardware with 10 qubits
• An end-to-end trainable hybrid classical-quantum classifier
• Advances in Quantum Reinforcement Learning
• Quantum embeddings for machine learning
• Classification with Quantum Machine Learning: A Survey
• Learning temporal data with variational quantum recurrent neural network
• TensorNetwork for Machine Learning
• Quantum enhancements for deep reinforcement learning in large spaces
• Generalized quantum circuit differentiation rules
• Markovian Quantum Neuroevolution for Machine Learning
• Entanglement and tensor network states
• Generative Tensor Network Classification Model for Supervised Machine Learning
• Natural Evolutionary Strategies for Variational Quantum Computation
• Quantum Architecture Search via Deep Reinforcement Learning
• Quantum semi-supervised generative adversarial network for enhanced data classification
• Hybrid Quantum-Classical Graph Convolutional Network
• Modeling Sequences with Quantum States: A Look Under the Hood
• Recurrent Quantum Neural Networks
• Quantum Quantile Mechanics: Solving Stochastic Differential Equations for Generating Time-Series
• Quantum Long Short-Term Memory
• QuClassi: A Hybrid Deep Neural Network Architecture based on Quantum State Fidelity
• Variational Quanvolutional Neural Networks with enhanced image encoding
• Framework for learning agents in quantum environments
• Quantum Convolutional Neural Networks for High Energy Physics Data Analysis
• QTN-VQC: An End-to-End Learning framework for Quantum Neural Networks
• Quantum Unsupervised and Supervised Learning on Superconducting Processors
• Evolving Neural Networks through a Reverse Encoding Tree
• Introduction to Quantum Reinforcement Learning: Theory and PennyLane-based Implementation
• Towards Building A Facial Identification System Using Quantum Machine Learning Techniques

Cited by in corpus (11)

• Quantum computing for finance
• Self-Correcting Quantum Many-Body Control using Reinforcement Learning with Tensor Networks
• Tensor networks for quantum machine learning
• Quantum Architecture Search with Meta-learning
• Quantum Deep Reinforcement Learning for Robot Navigation Tasks
• Optimizing quantum noise-induced reservoir computing for nonlinear and chaotic time series prediction
• Using Differential Evolution to avoid local minima in Variational Quantum Algorithms
• Hybrid actor-critic algorithm for quantum reinforcement learning at CERN beam lines
• Transfer Learning Analysis of Variational Quantum Circuits
• The Cytnx Library for Tensor Networks
• Quantum-Classical Autoencoder Architectures for End-to-End Radio Communication