Global optimization of quantum dynamics with AlphaZero deep exploration

Abstract

While a large number of algorithms for optimizing quantum dynamics for different objectives have been developed, a common limitation is the reliance on good initial guesses, being either random or based on heuristics and intuitions. Here we implement a tabula rasa deep quantum exploration version of the Deepmind AlphaZero algorithm for systematically averting this limitation. AlphaZero employs a deep neural network in conjunction with deep lookahead in a guided tree search, which allows for predictive hidden variable approximation of the quantum parameter landscape. To emphasize transferability, we apply and benchmark the algorithm on three classes of control problems using only a single common set of algorithmic hyperparameters. AlphaZero achieves substantial improvements in both the quality and quantity of good solution clusters compared to earlier methods. It is able to spontaneously learn unexpected hidden structure and global symmetry in the solutions, going beyond even human heuristics.

arXiv comment11 pages, 6 figures

References in corpus (46)

• Continuous control with deep reinforcement learning
• Quantum Computing in the NISQ era and beyond
• Quantum Machine Learning
• Charge insensitive qubit design derived from the Cooper pair box
• Hardware-efficient Variational Quantum Eigensolver for Small Molecules and Quantum Magnets
• Quantum Simulation
• Solving the Quantum Many-Body Problem with Artificial Neural Networks
• Evolution Strategies as a Scalable Alternative to Reinforcement Learning
• Training Schrödinger's cat: quantum optimal control
• Synthesis of Quantum Logic Circuits
• Procedure for systematically tuning up crosstalk in the cross resonance gate
• A simple all-microwave entangling gate for fixed-frequency superconducting qubits
• Quantum Theory From Five Reasonable Axioms
• Second order gradient ascent pulse engineering
• Comparing, Optimising and Benchmarking Quantum Control Algorithms in a Unifying Programming Framework
• Using Sideband Transitions for Two-Qubit Operations in Superconducting Circuits
• Benchmarking quantum control methods on a 12-qubit system
• Effective Hamiltonian models of the cross-resonance gate
• Microwave-induced coupling of superconducting qubits
• Simple random search provides a competitive approach to reinforcement learning
• Efficient synthesis of universal Repeat-Until-Success circuits
• Digital coherent control of a superconducting qubit
• Deep Reinforcement Learning for Quantum Gate Control
• Optimal control methods for fast time-varying Hamiltonians
• Exploring the Quantum Speed Limit with Computer Games
• Are there traps in quantum control landscapes?
• Generalizable control for quantum parameter estimation through reinforcement learning
• Evolutionary Algorithms for Hard Quantum Control
• Remote optimization of an ultra-cold atoms experiment by experts and citizen scientists
• Charting the circuit QED design landscape using optimal control theory
• Quantum Optimal Control in a Chopped Basis: Applications in Control of Bose-Einstein Condensates
• Quantum Control Landscapes: A Closer Look
• Automatic spin-chain learning to explore the quantum speed limit
• Exploring the trade-off between fidelity- and time-optimal control of quantum unitary transformations
• Reinforcement learning for autonomous preparation of Floquet-engineered states: Inverting the quantum Kapitza oscillator
• Glassy Phase of Optimal Quantum Control
• Optimal Qubit Control Using Single-Flux Quantum Pulses
• Measurement-based adaptation protocol with quantum reinforcement learning
• Optimized cross-resonance gate for coupled transmon systems
• Scalable Hardware-Efficient Qubit Control with Single Flux Quantum Pulse Sequences
• Linear and logarithmic time compositions of quantum many-body operators
• Broken symmetry in a two-qubit quantum control landscape
• Tunable coupling between three qubits as a building block for a superconducting quantum computer
• Optimal generation of Fock states in a weakly nonlinear oscillator
• Role of control constraints in quantum optimal control
• Computational Complexity and Phase Transitions

Cited by in corpus (60)

• On scientific understanding with artificial intelligence
• Model-Free Quantum Control with Reinforcement Learning
• Identifying optimal cycles in quantum thermal machines with reinforcement-learning
• Reinforcement Learning for Many-Body Ground-State Preparation Inspired by Counterdiabatic Driving
• Integrated tool-set for Control, Calibration and Characterization of quantum devices applied to superconducting qubits
• Deep Reinforcement Learning for Quantum State Preparation with Weak Nonlinear Measurements
• Deep Reinforcement Learning Control of Quantum Cartpoles
• Reinforcement learning for optimization of variational quantum circuit architectures
• Quantum enhancements for deep reinforcement learning in large spaces
• Robust Quantum Control in Games: an Adversarial Learning Approach
• Breaking Adiabatic Quantum Control with Deep Learning
• Optimal control of quantum thermal machines using machine learning
• Reinforcement learning optimization of the charging of a Dicke quantum battery
• Faster State Preparation across Quantum Phase Transition Assisted by Reinforcement Learning
• Chess AI: Competing Paradigms for Machine Intelligence
• Reinforcement learning of rare diffusive dynamics
• Pareto-optimal cycles for power, efficiency and fluctuations of quantum heat engines using reinforcement learning
• Policy Gradient based Quantum Approximate Optimization Algorithm
• Quantum optimal control with quantum computers: an hybrid algorithm featuring machine learning optimization
• Hessian-based optimization of constrained quantum control
• Deep Q-learning decoder for depolarizing noise on the toric code
• Predicting quantum dynamical cost landscapes with deep learning
• Designing Quantum Annealing Schedules using Bayesian Optimization
• Deep reinforcement learning for quantum Hamiltonian engineering
• Crowdsourcing human common sense for quantum control
• State leakage during fast decay and control of a superconducting transmon qubit
• Approximate Dynamics Lead to More Optimal Control: Efficient Exact Derivatives
• Experimental error suppression in Cross-Resonance gates via multi-derivative pulse shaping
• Beyond Games: A Systematic Review of Neural Monte Carlo Tree Search Applications
• Statistically Characterising Robustness and Fidelity of Quantum Controls and Quantum Control Algorithms
• Fast, high precision dynamics in quantum optimal control theory
• Taming quantum systems: A tutorial for using shortcuts-to-adiabaticity, quantum optimal control, and reinforcement learning
• Speeding up qubit control with bipolar single-flux-quantum pulse sequences
• Detection of entangled states supported by reinforcement learning
• Model-free optimization of power/efficiency tradeoffs in quantum thermal machines using reinforcement learning
• Reinforcement learning pulses for transmon qubit entangling gates
• Assessing Game Balance with AlphaZero: Exploring Alternative Rule Sets in Chess
• Hybrid discrete-continuous compilation of trapped-ion quantum circuits with deep reinforcement learning
• Ultrafast critical ground state preparation via bang-bang protocols
• Noise-Robust End-to-End Quantum Control using Deep Autoregressive Policy Networks
• Exploring the Optimal Cycle for Quantum Heat Engine using Reinforcement Learning
• Practical implications of SFQ-based two-qubit gates
• Classification of Equation of State in Relativistic Heavy-Ion Collisions Using Deep Learning
• Gradient projection method for constrained quantum control
• Neural network agent playing spin Hamiltonian games on a quantum computer
• Quantum optimal control in quantum technologies. Strategic report on current status, visions and goals for research in Europe
• Reinforcement Learning vs. Gradient-Based Optimisation for Robust Energy Landscape Control of Spin-1/2 Quantum Networks
• Unsupervised interpretable learning of topological indices invariant under permutations of atomic bands
• Parametrized multiqubit gate design for neutral-atom based quantum platforms
• Artificially intelligent Maxwell's demon for optimal control of open quantum systems
• Dropout is all you need: robust two-qubit gate with reinforcement learning
• Quantum Circuit Distillation and Compression
• Effect of Decoherence for Gate Operations on a Superconducting Bosonic Qubit
• The Value of Chess Squares
• Qubit Energy Tuner Based on Single Flux Quantum Circuits
• A Monte Carlo Tree Search approach to QAOA: finding a needle in the haystack
• Bayesian optimization of non-classical optomechanical correlations
• The Quantum Cartpole: A benchmark environment for non-linear reinforcement learning
• Globally optimal control of quantum dynamics
• Hybrid Reward-Driven Reinforcement Learning for Efficient Quantum Circuit Synthesis