When does reinforcement learning stand out in quantum control? A comparative study on state preparation

Abstract

Reinforcement learning has been widely used in many problems, including quantum control of qubits. However, such problems can, at the same time, be solved by traditional, non-machine-learning methods, such as stochastic gradient descent and Krotov algorithms, and it remains unclear which one is most suitable when the control has specific constraints. In this work, we perform a comparative study on the efficacy of three reinforcement learning algorithms: tabular Q-learning, deep Q-learning, and policy gradient, as well as two non-machine-learning methods: stochastic gradient descent and Krotov algorithms, in the problem of preparing a desired quantum state. We found that overall, the deep Q-learning and policy gradient algorithms outperform others when the problem is discretized, e.g. allowing discrete values of control, and when the problem scales up. The reinforcement learning algorithms can also adaptively reduce the complexity of the control sequences, shortening the operation time and improving the fidelity. Our comparison provides insights into the suitability of reinforcement learning in quantum control problems.

arXiv comment8+6 pages, 5+5 figures

References in corpus (34)

• Continuous control with deep reinforcement learning
• Solving the Quantum Many-Body Problem with Artificial Neural Networks
• Asynchronous Methods for Deep Reinforcement Learning
• Machine learning phases of matter
• Discovering Phase Transitions with Unsupervised Learning
• Roads towards fault-tolerant universal quantum computation
• Optimal Control at the Quantum Speed Limit
• Quantum-enhanced machine learning
• Active learning machine learns to create new quantum experiments
• Hybrid Quantum-Classical Approach to Quantum Optimal Control
• Quantum Loop Topography for Machine Learning
• Optimal quantum control using randomized benchmarking
• Flow Navigation by Smart Microswimmers via Reinforcement Learning
• Ultrafast optical rotations of electron spins in quantum dots
• Machine Learning for Precise Quantum Measurement
• Analytically solvable driven time-dependent two-level quantum systems
• Fidelity-based Probabilistic Q-learning for Control of Quantum Systems
• Optimizing Quantum Error Correction Codes with Reinforcement Learning
• Communication at the quantum speed limit along a spin chain
• Self-guided quantum tomography
• Efficient Discrete Approximations of Quantum Gates
• Robust control of quantum gates via sequential convex programming
• Generalizable control for quantum parameter estimation through reinforcement learning
• Quantum error correction for the toric code using deep reinforcement learning
• Machine Learning Many-Body Localization: Search for the Elusive Nonergodic Metal
• Branes with Brains: Exploring String Vacua with Deep Reinforcement Learning
• Optimal quantum control of Bose-Einstein condensates in magnetic microtraps: Comparison of GRAPE and Krotov optimization schemes
• Automatic spin-chain learning to explore the quantum speed limit
• Reinforcement learning for autonomous preparation of Floquet-engineered states: Inverting the quantum Kapitza oscillator
• Glassy Phase of Optimal Quantum Control
• A Data-Driven Gradient Algorithm for High-Precision Quantum Control
• Measurement-based adaptation protocol with quantum reinforcement learning
• Optically induced rotation of a quantum dot exciton spin
• Generation of ice states through deep reinforcement learning

Cited by in corpus (70)

• Entanglement Devised Barren Plateau Mitigation
• Quantum information processing with bosonic qubits in circuit QED
• Machine Learning for Quantum Matter
• Model-Free Quantum Control with Reinforcement Learning
• Experimental Deep Reinforcement Learning for Error-Robust Gateset Design on a Superconducting Quantum Computer
• Identifying optimal cycles in quantum thermal machines with reinforcement-learning
• Coherent Transport of Quantum States by Deep Reinforcement Learning
• Measurement Based Feedback Quantum Control With Deep Reinforcement Learning for Double-well Non-linear Potential
• Reinforcement Learning assisted Quantum Optimization
• A Tutorial on Optimal Control and Reinforcement Learning methods for Quantum Technologies
• Reinforcement learning approach to non-equilibrium quantum thermodynamics
• Quantum machine learning and quantum biomimetics: A perspective
• Breaking Adiabatic Quantum Control with Deep Learning
• Reinforcement learning optimization of the charging of a Dicke quantum battery
• Pareto-optimal cycles for power, efficiency and fluctuations of quantum heat engines using reinforcement learning
• Quantum imaginary time evolution steered by reinforcement learning
• Optimal control of families of quantum gates
• Generalizable control for multiparameter quantum metrology
• Cryogenic in-memory computing using magnetic topological insulators
• Deep reinforcement learning for quantum Hamiltonian engineering
• Quantum kernels to learn the phases of quantum matter
• Generic detection-based error-mitigation using quantum autoencoders
• Machine-learning assisted quantum control in random environment
• An automated approach for consecutive tuning of quantum dot arrays
• Quantum reinforcement learning in continuous action space
• Machine learning and control engineering: The model-free case
• Optimal quantum control via genetic algorithms for quantum state engineering in driven-resonator mediated networks
• Quantum Logic Gate Synthesis as a Markov Decision Process
• Optimal Control in Disordered Quantum Systems
• Active Learning Approach to Optimization of Experimental Control
• Stochastic learning control of adiabatic speedup in a non-Markovian open qutrit system
• Data-Centric Machine Learning in Quantum Information Science
• Model-free optimization of power/efficiency tradeoffs in quantum thermal machines using reinforcement learning
• Exploring Quantum Control Landscape and Solution Space Complexity through Dimensionality Reduction & Optimization Algorithms
• Machine Learning for Estimation and Control of Quantum Systems
• Universal quantum state preparation via revised greedy algorithm
• Solving Rubik's Cube via Quantum Mechanics and Deep Reinforcement Learning
• Approximations in transmon simulation
• Accelerated Magnonic Motional Cooling with Deep Reinforcement Learning
• Fast generation of entanglement between coupled spins using optimization and deep learning methods
• Push-Pull Optimization of Quantum Controls
• Exploring the Optimal Cycle for Quantum Heat Engine using Reinforcement Learning
• Enhanced quantum state preparation via stochastic prediction of neural network
• Simulating long-range hopping with periodically-driven superconducting qubits
• Reinforcement Learning for Rotation Sensing with Ultracold Atoms in an Optical Lattice
• Quantum optimal control in quantum technologies. Strategic report on current status, visions and goals for research in Europe
• Applications of model-aware reinforcement learning in Bayesian quantum metrology
• Model-aware reinforcement learning for high-performance Bayesian experimental design in quantum metrology
• Quantum Circuit Discovery for Fault-Tolerant Logical State Preparation with Reinforcement Learning
• Improving robustness of quantum feedback control with reinforcement learning
• Tangent space generators of matrix product states and exact Floquet quantum scars
• Optimized control for high-fidelity state transmission in open systems
• Persistent Ballistic Entanglement Spreading with Optimal Control in Quantum Spin Chains
• Fast quantum gate design with deep reinforcement learning using real-time feedback on readout signals
• Artificially intelligent Maxwell's demon for optimal control of open quantum systems
• Dropout is all you need: robust two-qubit gate with reinforcement learning
• Estimation of the geometric measure of entanglement with Wehrl Moments through Artificial Neural Networks
• Policy Gradients using Variational Quantum Circuits
• Optimizing measurement-based cooling by reinforcement learning
• No-Collapse Accurate Quantum Feedback Control via Conditional State Tomography
• Modularized and Scalable Compilation for Double Quantum Dot Quatum Computing
• Binary Quantum Control Optimization with Uncertain Hamiltonians
• Trade-off between Bagging and Boosting for quantum separability-entanglement classification
• Quantum data generation in a denoising model with multiscale entanglement renormalization network
• Quantum state-preparation control in noisy environment via most-likely paths
• Reinforcement Learning with Neural Networks for Quantum Multiple Hypothesis Testing
• An almost deterministic cooling by measurements
• Approximate quantum gates compiling with self-navigation algorithm
• Bounding fidelity in quantum feedback control: theory and applications to Dicke state preparation
• Molecular Quantum Control Algorithm Design by Reinforcement Learning