Experimental quantum speed-up in reinforcement learning agents

Abstract

Increasing demand for algorithms that can learn quickly and efficiently has led to a surge of development within the field of artificial intelligence (AI). An important paradigm within AI is reinforcement learning (RL), where agents interact with environments by exchanging signals via a communication channel. Agents can learn by updating their behaviour based on obtained feedback. The crucial question for practical applications is how fast agents can learn to respond correctly. An essential figure of merit is therefore the learning time. While various works have made use of quantum mechanics to speed up the agent's decision-making process, a reduction in learning time has not been demonstrated yet. Here we present a RL experiment where the learning of an agent is boosted by utilizing a quantum communication channel with the environment. We further show that the combination with classical communication enables the evaluation of such an improvement, and additionally allows for optimal control of the learning progress. This novel scenario is therefore demonstrated by considering hybrid agents, that alternate between rounds of quantum and classical communication. We implement this learning protocol on a compact and fully tunable integrated nanophotonic processor. The device interfaces with telecom-wavelength photons and features a fast active feedback mechanism, allowing us to demonstrate the agent's systematic quantum advantage in a setup that could be readily integrated within future large-scale quantum communication networks.

10 pages, 4 figures

References in corpus (29)

• Supplementary information for "Quantum supremacy using a programmable superconducting processor"
• The Quantum Internet
• Quantum Mechanics helps in searching for a needle in a haystack
• Advances in Quantum Metrology
• Deep Learning with Coherent Nanophotonic Circuits
• Detecting Single Infrared Photons with 93% System Efficiency
• Tight bounds on quantum searching
• Quantum transport simulations in a programmable nanophotonic processor
• Quantum Internet: Networking Challenges in Distributed Quantum Computing
• Quantum-enhanced machine learning
• Quantum information transfer using photons
• Phase-stable source of polarization-entangled photons using a polarization Sagnac interferometer
• Large-scale integration of near-indistinguishable artificial atoms in hybrid photonic circuits
• Quantum reinforcement learning
• Active learning machine learns to create new quantum experiments
• Automated Search for new Quantum Experiments
• Fixed-point quantum search with an optimal number of queries
• Quantum speedup for active learning agents
• Optimizing Quantum Error Correction Codes with Reinforcement Learning
• Machine-learning-assisted correction of correlated qubit errors in a topological code
• Deep neural decoders for near term fault-tolerant experiments
• Reconstruction of a Photonic Qubit State with Reinforcement Learning
• Learning to Perform Physics Experiments via Deep Reinforcement Learning
• Scalable Neural Network Decoders for Higher Dimensional Quantum Codes
• Experimental few-copy multi-particle entanglement detection
• Quantum-enhanced deliberation of learning agents using trapped ions
• Speeding-up the decision making of a learning agent using an ion trap quantum processor
• Photonic architecture for reinforcement learning
• Operationally meaningful representations of physical systems in neural networks

Cited by in corpus (58)

• Strong quantum computational advantage using a superconducting quantum processor
• Challenges and Opportunities in Quantum Machine Learning
• AI for Next Generation Computing: Emerging Trends and Future Directions
• A 12-mode Universal Photonic Processor for Quantum Information Processing
• Hybrid quantum neural network for drug response prediction
• Experimental quantum adversarial learning with programmable superconducting qubits
• Quantum Machine Learning: from physics to software engineering
• Theory for Equivariant Quantum Neural Networks
• Entanglement-Based Quantum Information Technology
• Theoretical Guarantees for Permutation-Equivariant Quantum Neural Networks
• A Tutorial on Optimal Control and Reinforcement Learning methods for Quantum Technologies
• Demonstration of algorithmic quantum speedup
• Generalized quantum circuit differentiation rules
• Self-Correcting Quantum Many-Body Control using Reinforcement Learning with Tensor Networks
• Hybrid quantum image classification and federated learning for hepatic steatosis diagnosis
• Hybrid quantum physics-informed neural networks for simulating computational fluid dynamics in complex shapes
• Quantum Reinforcement Learning: the Maze problem
• Quantum Deep Reinforcement Learning for Robot Navigation Tasks
• Quantum imaginary time evolution steered by reinforcement learning
• Information processing at the speed of light
• A duplication-free quantum neural network for universal approximation
• Solving the time-complexity problem and tuning the performance of quantum reservoir computing by artificial memory restriction
• Quantum reinforcement learning in continuous action space
• Experimental demonstration of Gaussian boson sampling with displacement
• Characterization of multi-mode linear optical networks
• Benefits of Open Quantum Systems for Quantum Machine Learning
• A hybrid classical-quantum approach to speed-up Q-learning
• Quantum Machine Learning Implementations: Proposals and Experiments
• What can we learn from quantum convolutional neural networks?
• Quantum Continual Learning Overcoming Catastrophic Forgetting
• Detection of entangled states supported by reinforcement learning
• Noise-Resistant Quantum State Compression Readout
• State-independent Nonadiabatic Geometric Quantum Gates
• Robust excitation of C-band quantum dots for quantum communication
• Quantum-accessible reinforcement learning beyond strictly epochal environments
• Performance analysis of a hybrid agent for quantum-accessible reinforcement learning
• Machine Learning for Estimation and Control of Quantum Systems
• Measurable Krylov Spaces and Eigenenergy Count in Quantum State Dynamics
• Evaluating the Resilience of Variational Quantum Algorithms to Leakage Noise
• Quantum reinforcement learning in the presence of thermal dissipation
• Demonstration of Algorithmic Quantum Speedup for an Abelian Hidden Subgroup Problem
• A high-fidelity and large-scale reconfigurable photonic processor for NISQ applications
• Framework for Learning and Control in the Classical and Quantum Domains
• Non-Classical Kernels in Continuous Variable Systems
• Preparing Quantum States by Measurement-feedback Control with Bayesian Optimization
• Unitary Evolutions Sourced By Interacting Quantum Memories: Closed Quantum Systems Directing Themselves Using Their State Histories
• Quantum Software Ecosystem Design
• Quantum Anomaly Detection with a Spin Processor in Diamond
• Quantum-secure multiparty deep learning
• Policy Gradients using Variational Quantum Circuits
• Engineering Quantum Reservoirs through Krylov Complexity, Expressivity and Observability
• Holographic photonic neuron
• Asymmetric quantum decision-making
• Optical Quantum Computing
• Post-Selections in AI and How to Avoid Them
• Energetic advantages for quantum agents in online execution of complex strategies
• On Quantum Perceptron Learning via Quantum Search
• Emergence of biased errors in imperfect photonic circuits