Quantum enhancements for deep reinforcement learning in large spaces

Abstract

In the past decade, the field of quantum machine learning has drawn significant attention due to the prospect of bringing genuine computational advantages to now widespread algorithmic methods. However, not all domains of machine learning have benefited equally from quantum enhancements. Notably, deep learning and reinforcement learning, despite their tremendous success in the classical domain, both individually and combined, remain relatively unaddressed by the quantum community. Arguably, one reason behind this is the systematic use in these domains of models and methods without prominent computational bottlenecks, leaving little room for quantum improvements. In this work, we study the state-of-the-art neural-network approaches for reinforcement learning with quantum enhancements in mind. We demonstrate the substantial learning advantage that models with a sampling bottleneck can provide over conventional neural network architectures in complex learning environments. These so-called energy-based models, like deep energy-based reinforcement learning, and deep projective simulation that we also introduce in this work, effectively allow to trade off learning performance for efficiency of computation. To alleviate the additional computational costs, we propose to leverage future and near-term quantum algorithms, resulting in overall more advantageous learning algorithms. This is achieved using cutting-edge and new quantum computing machinery to speed-up classical sampling methods and by employing generalized models to gain an additional quantum advantage.

arXiv commentSignificant number of new analyses and results

References in corpus (46)

• Quantum Computing in the NISQ era and beyond
• Quantum Machine Learning
• Supervised learning with quantum enhanced feature spaces
• Quantum support vector machine for big data classification
• Quantum machine learning in feature Hilbert spaces
• Quantum Amplitude Amplification and Estimation
• Quantum principal component analysis
• An introduction to quantum machine learning
• Deep Reinforcement Learning for De-Novo Drug Design
• Circuit-centric quantum classifiers
• Hamiltonian Simulation by Qubitization
• Quantum Data Fitting
• Practical Quantum Error Mitigation for Near-Future Applications
• Quantum machine learning: a classical perspective
• Quantum-enhanced machine learning
• Quantum Boltzmann Machine
• Towards Quantum Machine Learning with Tensor Networks
• Quantum Metropolis Sampling
• Reinforcement Learning in Different Phases of Quantum Control
• Active learning machine learns to create new quantum experiments
• Search via Quantum Walk
• Differentiable Learning of Quantum Circuit Born Machine
• Reinforcement Learning with Neural Networks for Quantum Feedback
• Fixed-point quantum search with an optimal number of queries
• Training of Quantum Circuits on a Hybrid Quantum Computer
• Quantum speedup for active learning agents
• Quantum Simulations of Classical Annealing Processes
• Tomography and Generative Data Modeling via Quantum Boltzmann Training
• Optimizing Quantum Error Correction Codes with Reinforcement Learning
• A Quantum-Quantum Metropolis Algorithm
• Variational Thermal Quantum Simulation via Thermofield Double States
• Speed-up via Quantum Sampling
• Machine learning for long-distance quantum communication
• Quantum error correction for the toric code using deep reinforcement learning
• Coherent Transport of Quantum States by Deep Reinforcement Learning
• Global optimization of quantum dynamics with AlphaZero deep exploration
• Quantum SDP-Solvers: Better upper and lower bounds
• Reinforcement learning for autonomous preparation of Floquet-engineered states: Inverting the quantum Kapitza oscillator
• Deep Reinforcement Learning Control of Quantum Cartpoles
• Applying quantum algorithms to constraint satisfaction problems
• Entanglement spectroscopy with a depth-two quantum circuit
• Setting up experimental Bell test with reinforcement learning
• Quantum-enhanced deliberation of learning agents using trapped ions
• Multicanonical MCMC for Sampling Rare Events
• Benchmarking projective simulation in navigation problems
• Faster quantum mixing for slowly evolving sequences of Markov chains

Cited by in corpus (17)

• Quantum computing for finance
• Quantum Computing for High-Energy Physics: State of the Art and Challenges. Summary of the QC4HEP Working Group
• Quantum agents in the Gym: a variational quantum algorithm for deep Q-learning
• Variational Quantum Reinforcement Learning via Evolutionary Optimization
• Understanding quantum machine learning also requires rethinking generalization
• Self-Correcting Quantum Many-Body Control using Reinforcement Learning with Tensor Networks
• Neural network-based prediction of the secret-key rate of quantum key distribution
• Quantum imaginary time evolution steered by reinforcement learning
• Quantum reinforcement learning in continuous action space
• Near-Optimal Quantum Algorithms for Multivariate Mean Estimation
• A hybrid classical-quantum approach to speed-up Q-learning
• Hybrid actor-critic algorithm for quantum reinforcement learning at CERN beam lines
• Neural quantum kernels: training quantum kernels with quantum neural networks
• Hybrid discrete-continuous compilation of trapped-ion quantum circuits with deep reinforcement learning
• Performance analysis of a hybrid agent for quantum-accessible reinforcement learning
• Satellite image classification with neural quantum kernels
• Cavity-Heisenberg spin-$j$ chain quantum battery and reinforcement learning optimization