Reinforcement Learning in Different Phases of Quantum Control

Abstract

The ability to prepare a physical system in a desired quantum state is central to many areas of physics such as nuclear magnetic resonance, cold atoms, and quantum computing. Yet, preparing states quickly and with high fidelity remains a formidable challenge. In this work we implement cutting-edge Reinforcement Learning (RL) techniques and show that their performance is comparable to optimal control methods in the task of finding short, high-fidelity driving protocol from an initial to a target state in non-integrable many-body quantum systems of interacting qubits. RL methods learn about the underlying physical system solely through a single scalar reward (the fidelity of the resulting state) calculated from numerical simulations of the physical system. We further show that quantum state manipulation, viewed as an optimization problem, exhibits a spin-glass-like phase transition in the space of protocols as a function of the protocol duration. Our RL-aided approach helps identify variational protocols with nearly optimal fidelity, even in the glassy phase, where optimal state manipulation is exponentially hard. This study highlights the potential usefulness of RL for applications in out-of-equilibrium quantum physics.

A legend for the videos referred to in the paper is available on https://mgbukov.github.io/RL_movies/

References in corpus (27)

• Solving the Quantum Many-Body Problem with Artificial Neural Networks
• Machine learning phases of matter
• Quantum Simulation of Antiferromagnetic Spin Chains in an Optical Lattice
• Quantum-enhanced machine learning
• QuSpin: a Python Package for Dynamics and Exact Diagonalisation of Quantum Many Body Systems part I: spin chains
• Active learning machine learns to create new quantum experiments
• Minimizing irreversible losses in quantum systems by local counter-diabatic driving
• Accelerated quantum control using superadiabatic dynamics in a solid-state lambda system
• Flow Navigation by Smart Microswimmers via Reinforcement Learning
• Reinforcement Learning with Neural Networks for Quantum Feedback
• Fidelity-based Probabilistic Q-learning for Control of Quantum Systems
• Optimizing Variational Quantum Algorithms using Pontryagin's Minimum Principle
• Time evolution algorithms for Matrix Product States and DMRG
• Adiabatic Perturbation Theory and Geometry of Periodically-Driven Systems
• Speedup for quantum optimal control from automatic differentiation based on graphics processing units
• High-Fidelity Single-Shot Toffoli Gate via Quantum Control
• Experimental Realization of a Quantum Integer-Spin Chain with Controllable Interactions
• Quantum brachistochrone curves as geodesics: obtaining accurate control protocols for time-optimal quantum gates
• A geometric approach to optimal nonequilibrium control: Minimizing dissipation in nanomagnetic spin systems
• Automatic spin-chain learning to explore the quantum speed limit
• A Data-Driven Gradient Algorithm for High-Precision Quantum Control
• Superadiabatic quantum state transfer in spin chains
• Optimal control of superconducting gmon qubits using Pontryagin's minimum principle: preparing a maximally entangled state with singular bang-bang protocols
• Assessing the significance of fidelity as a figure of merit in quantum state reconstruction of discrete and continuous variable systems
• Broken symmetry in a two-qubit quantum control landscape
• Superadiabatic driving of a three-level quantum system
• Clustering in Hilbert space of a quantum optimization problem

Cited by in corpus (195)

• Machine learning and the physical sciences
• Quantum Approximate Optimization Algorithm: Performance, Mechanism, and Implementation on Near-Term Devices
• QuSpin: a Python Package for Dynamics and Exact Diagonalisation of Quantum Many Body Systems. Part II: bosons, fermions and higher spins
• Introduction to the Pontryagin Maximum Principle for Quantum Optimal Control
• Artificial Intelligence and Machine Learning for Quantum Technologies
• Machine Learning for Quantum Matter
• Capacity and quantum geometry of parametrized quantum circuits
• Model-Free Quantum Control with Reinforcement Learning
• Experimental Deep Reinforcement Learning for Error-Robust Gateset Design on a Superconducting Quantum Computer
• Learning Robust and High-Precision Quantum Controls
• Krotov: A Python implementation of Krotov's method for quantum optimal control
• Machine learning of phase transitions in the percolation and XY models
• Quantum error correction for the toric code using deep reinforcement learning
• Geometric Speed Limit of Accessible Many-Body State Preparation
• Branes with Brains: Exploring String Vacua with Deep Reinforcement Learning
• Reinforcement Learning for Many-Body Ground-State Preparation Inspired by Counterdiabatic Driving
• Unsupervised machine learning of topological phase transitions from experimental data
• Counterdiabatic Optimised Local Driving
• Deep Reinforcement Learning for Quantum State Preparation with Weak Nonlinear Measurements
• Modelling Non-Markovian Quantum Processes with Recurrent Neural Networks
• Reinforcement Learning assisted Quantum Optimization
• Precise measurement of quantum observables with neural-network estimators
• Reinforcement learning for autonomous preparation of Floquet-engineered states: Inverting the quantum Kapitza oscillator
• Glassy Phase of Optimal Quantum Control
• Deep Reinforcement Learning Control of Quantum Cartpoles
• Detecting quantum entanglement with unsupervised learning
• Measurement-based adaptation protocol with quantum reinforcement learning
• Machine Learning meets Quantum Foundations: A Brief Survey
• Calibration of quantum sensors by neural networks
• Introduction to Theoretical and Experimental aspects of Quantum Optimal Control
• A Reinforcement Learning approach for Quantum State Engineering
• Reinforcement learning approach to non-equilibrium quantum thermodynamics
• Quantum enhancements for deep reinforcement learning in large spaces
• Approximate Autonomous Quantum Error Correction with Reinforcement Learning
• Quantum optimal control of multi-level dissipative quantum systems with Reinforcement Learning
• Quantum machine learning and quantum biomimetics: A perspective
• A differentiable programming method for quantum control
• Breaking Adiabatic Quantum Control with Deep Learning
• A reinforcement learning approach to rare trajectory sampling
• Reinforcement Learning for Digital Quantum Simulation
• Improving the dynamics of quantum sensors with reinforcement learning
• Drawing Phase Diagrams of Random Quantum Systems by Deep Learning the Wave Functions
• Optimal control of quantum thermal machines using machine learning
• Reinforcement learning optimization of the charging of a Dicke quantum battery
• Faster State Preparation across Quantum Phase Transition Assisted by Reinforcement Learning
• Divergence of predictive model output as indication of phase transitions
• Predicting quantum advantage by quantum walk with convolutional neural networks
• Gradient Ascent Pulse Engineering with Feedback
• Quantum Adiabatic Algorithm Design using Reinforcement Learning
• Classifying global state preparation via deep reinforcement learning
• Reinforcement learning of rare diffusive dynamics
• Pareto-optimal cycles for power, efficiency and fluctuations of quantum heat engines using reinforcement learning
• Neural networks for detecting multimode Wigner-negativity
• Optimal quantum control with poor statistics
• Approaching the adiabatic timescale with machine-learning
• Automated machine learning for secure key rate in discrete-modulated continuous-variable quantum key distribution
• Role of stochastic noise and generalization error in the time propagation of neural-network quantum states
• Unsupervised learning using topological data augmentation
• Deep Q-learning decoder for depolarizing noise on the toric code
• Quantum State Optimization and Computational Pathway Evaluation for Gate-Model Quantum Computers
• Hessian-based optimization of constrained quantum control
• Bilinear dynamic mode decomposition for quantum control
• Predicting quantum dynamical cost landscapes with deep learning
• Optimal control of families of quantum gates
• Robust quantum gates using smooth pulses and physics-informed neural networks
• Photonic architecture for reinforcement learning
• Optimal steering of matrix product states and quantum many-body scars
• Generalizable control for multiparameter quantum metrology
• Extreme Spin Squeezing from Deep Reinforcement Learning
• Time-optimal control of a dissipative qubit
• Phase Detection with Neural Networks: Interpreting the Black Box
• Deep reinforcement learning for universal quantum state preparation via dynamic pulse control
• Analyzing the impact of time-correlated noise on zero-noise extrapolation
• Machine Learning and Quantum Devices
• Quantum-tailored machine-learning characterization of a superconducting qubit
• Stochastic Estimation of Dynamical Variables
• Deep reinforcement learning for quantum Hamiltonian engineering
• Protocol Discovery for the Quantum Control of Majoranas by Differentiable Programming and Natural Evolution Strategies
• Modeling and Control of a Reconfigurable Photonic Circuit using Deep Learning
• Application of Pontryagin's Minimum Principle to Grover's Quantum Search Problem
• Reinforcement learning for semi-autonomous approximate quantum eigensolver
• Real-time calibration of coherent-state receivers: learning by trial and error
• K-GRAPE: A Krylov Subspace approach for the efficient control of quantum many-body dynamics
• Generalization of Quantum Machine Learning Models Using Quantum Fisher Information Metric
• Measurement-based cooling of a nonlinear mechanical resonator
• Preparation of ordered states in ultra-cold gases using Bayesian optimization
• Machine-learning assisted quantum control in random environment
• Universal and optimal coin sequences for high entanglement generation in 1D discrete time quantum walks
• Variational quantum dynamics of two-dimensional rotor models
• Demonstration of a programmable optical lattice atom interferometer
• Efficient Paths for Local Counterdiabatic Driving
• Quantum reinforcement learning in continuous action space
• Networked Quantum Services
• Control of Stochastic Quantum Dynamics by Differentiable Programming
• Accurate phonon blockade detector composed of a quadratically coupled optomechanical system
• Inhomogeneous quenches as state preparation in two-dimensional conformal field theories
• Cooling arbitrary near-critical systems using hyperbolic quenches
• Deep reinforcement learning for robust quantum optimization
• Generation and storage of spin squeezing via learning-assisted optimal control
• Fast, high precision dynamics in quantum optimal control theory
• Taming quantum systems: A tutorial for using shortcuts-to-adiabaticity, quantum optimal control, and reinforcement learning
• Optimal Control in Disordered Quantum Systems
• Active Learning Approach to Optimization of Experimental Control
• Drawing together control landscape and tomography principles
• Detection of entangled states supported by reinforcement learning
• Counterdiabatic optimized driving in quantum phase sensitive models
• Data-Centric Machine Learning in Quantum Information Science
• Mitigated barren plateaus in the time-nonlocal optimization of analog quantum-algorithm protocols
• Continuous quantum gate sets and pulse class meta-optimization
• Minimum time generation of a uniform superposition in a qubit with only transverse field control
• Ultrafast critical ground state preparation via bang-bang protocols
• Hybrid discrete-continuous compilation of trapped-ion quantum circuits with deep reinforcement learning
• Deep recurrent networks predicting the gap evolution in adiabatic quantum computing
• A dark state of Chern bands: Designing flat bands with higher Chern number
• Prospects of reinforcement learning for the simultaneous damping of many mechanical modes
• Robust optimization for quantum reinforcement learning control using partial observations
• Deep reinforcement learning for preparation of thermal and prethermal quantum states
• Certificates of quantum many-body properties assisted by machine learning
• Approximations in transmon simulation
• Solving Rubik's Cube via Quantum Mechanics and Deep Reinforcement Learning
• Creation and manipulation of quantized vortices in Bose-Einstein condensates using reinforcement learning
• Random coordinate descent: a simple alternative for optimizing parameterized quantum circuits
• Identifying nonclassicality from experimental data using artificial neural networks
• High-dimensional reinforcement learning for optimization and control of ultracold quantum gases
• Exploring the Optimal Cycle for Quantum Heat Engine using Reinforcement Learning
• Accelerated Magnonic Motional Cooling with Deep Reinforcement Learning
• Quantum generative model for sampling many-body spectral functions
• Stochastic optimal control formalism for an open quantum system
• Quantum reinforcement learning in the presence of thermal dissipation
• Simulating long-range hopping with periodically-driven superconducting qubits
• Toward a Theory of Phase Transitions in Quantum Control Landscapes
• Probing Criticality in Quantum Spin Chains with Neural Networks
• Cavity-Heisenberg spin-$j$ chain quantum battery and reinforcement learning optimization
• Designing fast quantum gates using optimal control with a reinforcement-learning ansatz
• The quantum annealing gap and quench dynamics in the exact cover problem
• Reinforcement learning assisted non-reciprocal optomechanical gyroscope
• Entanglement engineering of optomechanical systems by reinforcement learning
• Reinforcement Learning Approach to Shortcuts between Thermodynamic States with Extra Constraints
• Reinforcement Learning for Rotation Sensing with Ultracold Atoms in an Optical Lattice
• Model-aware reinforcement learning for high-performance Bayesian experimental design in quantum metrology
• Quantum circuit compilation with quantum computers
• Neural network agent playing spin Hamiltonian games on a quantum computer
• Time-Optimal Quantum Driving by Variational Circuit Learning
• Reinforcement Learning vs. Gradient-Based Optimisation for Robust Energy Landscape Control of Spin-1/2 Quantum Networks
• Hard instance learning for quantum adiabatic prime factorization
• Quantum optimal control in quantum technologies. Strategic report on current status, visions and goals for research in Europe
• Scalable Parameter Design for Superconducting Quantum Circuits with Graph Neural Networks
• Applications of model-aware reinforcement learning in Bayesian quantum metrology
• Learning quantum phases via single-qubit disentanglement
• Improving robustness of quantum feedback control with reinforcement learning
• Quantum Circuit Discovery for Fault-Tolerant Logical State Preparation with Reinforcement Learning
• Tangent space generators of matrix product states and exact Floquet quantum scars
• Spin-qubit noise spectroscopy from randomized benchmarking by supervised learning
• Dual-Capability Machine Learning Models for Quantum Hamiltonian Parameter Estimation and Dynamics Prediction
• Controlling nonergodicity in quantum many-body systems by reinforcement learning
• Persistent Ballistic Entanglement Spreading with Optimal Control in Quantum Spin Chains
• Fast quantum gate design with deep reinforcement learning using real-time feedback on readout signals
• Coarse-grained spectral projection (CGSP): a deep learning-assisted approach to quantum unitary dynamics
• Dropout is all you need: robust two-qubit gate with reinforcement learning
• Artificially intelligent Maxwell's demon for optimal control of open quantum systems
• Switching Time Optimization for Binary Quantum Optimal Control
• Hamiltonian Switching Control of Noisy Bipartite Qubit Systems
• Echo-evolution data generation for quantum error mitigation via neural networks
• Adaptive Random Quantum Eigensolver
• Control of open quantum systems: Manipulation of a qubit coupled to a thermal bath by an external driving field
• Charging a quantum spin network towards Heisenberg-limited precision
• Fast State Stabilization using Deep Reinforcement Learning for Measurement-based Quantum Feedback Control
• Learning a quantum computer's capability
• Spreading entanglement through pairwise exchange interactions
• No-Collapse Accurate Quantum Feedback Control via Conditional State Tomography
• Topological Phase Transitions in a Constrained Two-Qubit Quantum Control Landscape
• Tackling the Challenges of Adding Pulse-level Support to a Heterogeneous HPCQC Software Stack: MQSS Pulse
• Fingerprint Recognition of Partial Discharge Signals in Deep Learning Enhanced Rydberg Atomic Sensors
• Optimizing measurement-based cooling by reinforcement learning
• Overcoming disorder in superconducting globally driven quantum computing
• Trade-off between Bagging and Boosting for quantum separability-entanglement classification
• Deep learning estimation of complex reverberant wave fields by a programmable metasurface
• Geometrical versus time-series representation of data in quantum control learning
• Improving Variational Counterdiabatic Driving with Weighted Actions and Computer Algebra
• Estimating the degree of non-Markovianity using variational quantum circuits
• Robust implicit quantum control of interacting spin chains
• Globally optimal control of quantum dynamics
• Hidden local adiabatic ramp in the modulated time evolution and the quantum approximate optimization algorithm
• Measurement-based cooling of many-body quantum systems
• Reinforcement-learning based matterwave interferometer in a shaken optical lattice
• Preparation of Many-body Ground States by Time Evolution with Variational Microscopic Magnetic Fields and Incomplete Interactions
• The Quantum Cartpole: A benchmark environment for non-linear reinforcement learning
• Hybrid Reward-Driven Reinforcement Learning for Efficient Quantum Circuit Synthesis
• Shortcuts to Analog Preparation of Non-Equilibrium Quantum Lakes
• Noise-Resilient Quantum Reinforcement Learning
• Optimal Work Extraction from Finite-Time Closed Quantum Dynamics
• Role of overparametrization in quantum approximate optimization
• Exploring fixed points and eigenstates of quantum systems with reinforcement learning
• Model-Free Quantum Stabilization via Finite-Difference Lyapunov Control
• Multiclass classification of dephasing channels