Reinforcement Learning with Neural Networks for Quantum Feedback

Abstract

Machine learning with artificial neural networks is revolutionizing science. The most advanced challenges require discovering answers autonomously. This is the domain of reinforcement learning, where control strategies are improved according to a reward function. The power of neural-network-based reinforcement learning has been highlighted by spectacular recent successes, such as playing Go, but its benefits for physics are yet to be demonstrated. Here, we show how a network-based "agent" can discover complete quantum-error-correction strategies, protecting a collection of qubits against noise. These strategies require feedback adapted to measurement outcomes. Finding them from scratch, without human guidance, tailored to different hardware resources, is a formidable challenge due to the combinatorially large search space. To solve this, we develop two ideas: two-stage learning with teacher/student networks and a reward quantifying the capability to recover the quantum information stored in a multi-qubit system. Beyond its immediate impact on quantum computation, our work more generally demonstrates the promise of neural-network-based reinforcement learning in physics.

arXiv comment7 pages maintext + methods + supplementary, 6 maintext figures; for related lectures, see: http://machine-learning-for-physicists.org

References in corpus (31)

• Quantum Machine Learning
• Solving the Quantum Many-Body Problem with Artificial Neural Networks
• Machine learning phases of matter
• Quantum Error Correction for Quantum Memories
• State preservation by repetitive error detection in a superconducting quantum circuit
• A Two Qubit Logic Gate in Silicon
• Demonstration of a small programmable quantum computer with atomic qubits
• Learning phase transitions by confusion
• A programmable two-qubit quantum processor in silicon
• Dynamically protected cat-qubits: a new paradigm for universal quantum computation
• Quantum error correction in a solid-state hybrid spin register
• Unconditional quantum teleportation between distant solid-state qubits
• Quantum autoencoders for efficient compression of quantum data
• Implementing a Universal Gate Set on a Logical Qubit Encoded in an Oscillator
• Active learning machine learns to create new quantum experiments
• Realization of Deterministic Quantum Teleportation with Solid State Qubits
• Comparing, Optimising and Benchmarking Quantum Control Algorithms in a Unifying Programming Framework
• Experimental demonstration of fault-tolerant state preparation with superconducting qubits
• Feedback control of a solid-state qubit using high-fidelity projective measurement
• Machine Learning for Precise Quantum Measurement
• A Neural Decoder for Topological Codes
• Fidelity-based Probabilistic Q-learning for Control of Quantum Systems
• Machine-learning-assisted correction of correlated qubit errors in a topological code
• Persistent control of a superconducting qubit by stroboscopic measurement feedback
• Studying Light-Harvesting Models with Superconducting Circuits
• Deep Neural Network Probabilistic Decoder for Stabilizer Codes
• An Efficient Algorithm for Optimizing Adaptive Quantum Metrology Processes
• Learning in Quantum Control: High-Dimensional Global Optimization for Noisy Quantum Dynamics
• Two-dimensional cavity grid for scalable quantum computation with superconducting circuits
• Using Recurrent Neural Networks to Optimize Dynamical Decoupling for Quantum Memory
• Adaptive quantum computation in changing environments using projective simulation

Cited by in corpus (144)

• Machine learning and the physical sciences
• Reinforcement Learning in Different Phases of Quantum Control
• Artificial Intelligence and Machine Learning for Quantum Technologies
• Differentiable Quantum Architecture Search
• Optimizing Quantum Error Correction Codes with Reinforcement Learning
• Machine Learning for Quantum Matter
• Model-Free Quantum Control with Reinforcement Learning
• Machine learning for long-distance quantum communication
• Variational Quantum Reinforcement Learning via Evolutionary Optimization
• Quantum Machine Learning: from physics to software engineering
• Neural Belief-Propagation Decoders for Quantum Error-Correcting Codes
• Quantum error correction for the toric code using deep reinforcement learning
• Geometric Speed Limit of Accessible Many-Body State Preparation
• Neural Predictor based Quantum Architecture Search
• Using a Recurrent Neural Network to Reconstruct Quantum Dynamics of a Superconducting Qubit from Physical Observations
• Reinforcement Learning for Many-Body Ground-State Preparation Inspired by Counterdiabatic Driving
• Measurement Based Feedback Quantum Control With Deep Reinforcement Learning for Double-well Non-linear Potential
• Neural-Network Decoders for Quantum Error Correction using Surface Codes:A Space Exploration of the Hardware Cost-Performance Trade-Offs
• Deep Reinforcement Learning for Quantum State Preparation with Weak Nonlinear Measurements
• Reinforcement Learning Decoders for Fault-Tolerant Quantum Computation
• Reinforcement Learning assisted Quantum Optimization
• Reinforcement learning for autonomous preparation of Floquet-engineered states: Inverting the quantum Kapitza oscillator
• Glassy Phase of Optimal Quantum Control
• Deep Reinforcement Learning Control of Quantum Cartpoles
• Physics-informed neural networks for quantum control
• A Tutorial on Optimal Control and Reinforcement Learning methods for Quantum Technologies
• Hamiltonian Learning for Quantum Error Correction
• Calibration of quantum sensors by neural networks
• Real-Time Decoding for Fault-Tolerant Quantum Computing: Progress, Challenges and Outlook
• A Reinforcement Learning approach for Quantum State Engineering
• Reinforcement learning approach to non-equilibrium quantum thermodynamics
• Quantum enhancements for deep reinforcement learning in large spaces
• Learning feedback control strategies for quantum metrology
• Approximate Autonomous Quantum Error Correction with Reinforcement Learning
• Quantum optimal control of multi-level dissipative quantum systems with Reinforcement Learning
• Markovian Quantum Neuroevolution for Machine Learning
• Quantum machine learning and quantum biomimetics: A perspective
• Deep reinforcement learning for quantum multiparameter estimation
• A differentiable programming method for quantum control
• Breaking Adiabatic Quantum Control with Deep Learning
• Conceptual understanding through efficient inverse-design of quantum optical experiments
• A reinforcement learning approach to rare trajectory sampling
• Reinforcement Learning for Digital Quantum Simulation
• Neural-Network Heuristics for Adaptive Bayesian Quantum Estimation
• Improving the dynamics of quantum sensors with reinforcement learning
• Optimal control of quantum thermal machines using machine learning
• Reinforcement learning optimization of the charging of a Dicke quantum battery
• Divergence of predictive model output as indication of phase transitions
• Predicting quantum advantage by quantum walk with convolutional neural networks
• Gradient Ascent Pulse Engineering with Feedback
• Reinforcement learning of rare diffusive dynamics
• Reinforcement learning for optimal error correction of toric codes
• Pareto-optimal cycles for power, efficiency and fluctuations of quantum heat engines using reinforcement learning
• Neural networks for detecting multimode Wigner-negativity
• Optimal quantum control with poor statistics
• Role of stochastic noise and generalization error in the time propagation of neural-network quantum states
• Unsupervised learning using topological data augmentation
• Deep Q-learning decoder for depolarizing noise on the toric code
• Robust quantum gates using smooth pulses and physics-informed neural networks
• Generalizable control for multiparameter quantum metrology
• Machine learning time-local generators of open quantum dynamics
• Roadmap on Nanoscale Magnetic Resonance Imaging
• Deep reinforcement learning for quantum Hamiltonian engineering
• Machine Learning and Quantum Devices
• Protocol Discovery for the Quantum Control of Majoranas by Differentiable Programming and Natural Evolution Strategies
• Operationally meaningful representations of physical systems in neural networks
• Deep Learning of Quantum Many-Body Dynamics via Random Driving
• Optimal Renormalization Group Transformation from Information Theory
• Quantum variational learning for quantum error-correcting codes
• Efficient and Robust Entanglement Generation with Deep Reinforcement Learning for Quantum Metrology
• Preparation of ordered states in ultra-cold gases using Bayesian optimization
• Real-time calibration of coherent-state receivers: learning by trial and error
• Machine-learning assisted quantum control in random environment
• Visualising energy landscapes through manifold learning
• Universal and optimal coin sequences for high entanglement generation in 1D discrete time quantum walks
• Demonstration of a programmable optical lattice atom interferometer
• Neural-network approach for identifying nonclassicality from click-counting data
• Continuous quantum error correction for evolution under time-dependent Hamiltonians
• Approximation of quantum control correction scheme using deep neural networks
• Deep reinforcement learning for key distribution based on quantum repeaters
• Generation of ice states through deep reinforcement learning
• Deep reinforcement learning for robust quantum optimization
• Taming quantum systems: A tutorial for using shortcuts-to-adiabaticity, quantum optimal control, and reinforcement learning
• Generation and storage of spin squeezing via learning-assisted optimal control
• Detection of entangled states supported by reinforcement learning
• Non-Markovian feedback for optimized quantum error correction
• Model-free optimization of power/efficiency tradeoffs in quantum thermal machines using reinforcement learning
• Classification and reconstruction of optical quantum states with deep neural networks
• Robust optimization for quantum reinforcement learning control using partial observations
• Deep learning of deformation-dependent conductance in thin films: nanobubbles in graphene
• Prospects of reinforcement learning for the simultaneous damping of many mechanical modes
• Machine Learning for Estimation and Control of Quantum Systems
• Continuous-time dynamics and error scaling of noisy highly-entangling quantum circuits
• Solving Rubik's Cube via Quantum Mechanics and Deep Reinforcement Learning
• Preparing Schrödinger cat states in a microwave cavity using a neural network
• Deep reinforcement learning for preparation of thermal and prethermal quantum states
• Optimizing ZX-Diagrams with Deep Reinforcement Learning
• Accelerated Magnonic Motional Cooling with Deep Reinforcement Learning
• Identifying nonclassicality from experimental data using artificial neural networks
• Creation and manipulation of quantized vortices in Bose-Einstein condensates using reinforcement learning
• Random telegraph signal analysis with a recurrent neural network
• Hybrid quantum-classical unsupervised data clustering based on the self-organizing feature map
• Quantum reinforcement learning in the presence of thermal dissipation
• Designing fast quantum gates using optimal control with a reinforcement-learning ansatz
• Cavity-Heisenberg spin-$j$ chain quantum battery and reinforcement learning optimization
• Entanglement engineering of optomechanical systems by reinforcement learning
• Reinforcement learning assisted non-reciprocal optomechanical gyroscope
• Bidirectional deep learning of polarization transfer in liquid crystals with application to quantum state preparation
• Privacy-preserving machine learning with tensor networks
• Neural network agent playing spin Hamiltonian games on a quantum computer
• Time-Optimal Quantum Driving by Variational Circuit Learning
• Reinforcement Learning for Rotation Sensing with Ultracold Atoms in an Optical Lattice
• Model-aware reinforcement learning for high-performance Bayesian experimental design in quantum metrology
• Applications of model-aware reinforcement learning in Bayesian quantum metrology
• Improving robustness of quantum feedback control with reinforcement learning
• Quantum Circuit Discovery for Fault-Tolerant Logical State Preparation with Reinforcement Learning
• Learning quantum phases via single-qubit disentanglement
• Deep reinforcement learning for feedback control in a collective flashing ratchet
• Artificially intelligent Maxwell's demon for optimal control of open quantum systems
• Parametrized multiqubit gate design for neutral-atom based quantum platforms
• Estimation of the geometric measure of entanglement with Wehrl Moments through Artificial Neural Networks
• Quantum feedback control with a transformer neural network architecture
• Quantum-classical reinforcement learning for decoding noisy classical parity information
• Enhancing Quantum Memory Lifetime with Measurement-Free Local Error Correction and Reinforcement Learning
• Learning a quantum computer's capability
• Policy Gradient Approach to Compilation of Variational Quantum Circuits
• Limitations of strong coupling in non-Markovian quantum thermometry
• Optimizing measurement-based cooling by reinforcement learning
• Qudit low-density parity-check codes
• No-Collapse Accurate Quantum Feedback Control via Conditional State Tomography
• Geometrical versus time-series representation of data in quantum control learning
• Emergence of global receptive fields capturing multipartite quantum correlations
• Estimating the degree of non-Markovianity using variational quantum circuits
• Bounding fidelity in quantum feedback control: theory and applications to Dicke state preparation
• Programming multi-level quantum gates in disordered computing reservoirs via machine learning and TensorFlow
• Adaptive Bayesian algorithm for achieving desired quantum transition
• Learning Feedback Mechanisms for Measurement-Based Variational Quantum State Preparation
• Exploring fixed points and eigenstates of quantum systems with reinforcement learning
• Autonomously Designed Pulses for Precise, Site-Selective Control of Atomic Qubits
• The Quantum Cartpole: A benchmark environment for non-linear reinforcement learning
• Reinforcement-learning based matterwave interferometer in a shaken optical lattice
• Automated discovery and optimization of autonomous quantum error correction codes for a general open quantum system
• Reinforcement Learning with Neural Networks for Quantum Multiple Hypothesis Testing
• Feedback cooling of fermionic atoms in optical lattices