Global optimization of quantum dynamics with AlphaZero deep exploration

Abstract

While a large number of algorithms for optimizing quantum dynamics for different objectives have been developed, a common limitation is the reliance on good initial guesses, being either random or based on heuristics and intuitions. Here we implement a tabula rasa deep quantum exploration version of the Deepmind AlphaZero algorithm for systematically averting this limitation. AlphaZero employs a deep neural network in conjunction with deep lookahead in a guided tree search, which allows for predictive hidden variable approximation of the quantum parameter landscape. To emphasize transferability, we apply and benchmark the algorithm on three classes of control problems using only a single common set of algorithmic hyperparameters. AlphaZero achieves substantial improvements in both the quality and quantity of good solution clusters compared to earlier methods. It is able to spontaneously learn unexpected hidden structure and global symmetry in the solutions, going beyond even human heuristics.

11 pages, 6 figures

References in corpus (9)

• Charge insensitive qubit design derived from the Cooper pair box
• Synthesis of Quantum Logic Circuits
• Using Sideband Transitions for Two-Qubit Operations in Superconducting Circuits
• Benchmarking quantum control methods on a 12-qubit system
• Microwave-induced coupling of superconducting qubits
• Efficient synthesis of universal Repeat-Until-Success circuits
• Deep Reinforcement Learning for Quantum Gate Control
• Generalizable control for quantum parameter estimation through reinforcement learning
• Linear and logarithmic time compositions of quantum many-body operators

Cited by in corpus (60)

• On scientific understanding with artificial intelligence
• Model-Free Quantum Control with Reinforcement Learning
• Identifying optimal cycles in quantum thermal machines with reinforcement-learning
• Reinforcement Learning for Many-Body Ground-State Preparation Inspired by Counterdiabatic Driving
• Integrated tool-set for Control, Calibration and Characterization of quantum devices applied to superconducting qubits
• Deep Reinforcement Learning for Quantum State Preparation with Weak Nonlinear Measurements
• Deep Reinforcement Learning Control of Quantum Cartpoles
• Reinforcement learning for optimization of variational quantum circuit architectures
• Quantum enhancements for deep reinforcement learning in large spaces
• Robust Quantum Control in Games: an Adversarial Learning Approach
• Breaking Adiabatic Quantum Control with Deep Learning
• Optimal control of quantum thermal machines using machine learning
• Reinforcement learning optimization of the charging of a Dicke quantum battery
• Faster State Preparation across Quantum Phase Transition Assisted by Reinforcement Learning
• Chess AI: Competing Paradigms for Machine Intelligence
• Reinforcement learning of rare diffusive dynamics
• Pareto-optimal cycles for power, efficiency and fluctuations of quantum heat engines using reinforcement learning
• Policy Gradient based Quantum Approximate Optimization Algorithm
• Quantum optimal control with quantum computers: an hybrid algorithm featuring machine learning optimization
• Hessian-based optimization of constrained quantum control
• Deep Q-learning decoder for depolarizing noise on the toric code
• Predicting quantum dynamical cost landscapes with deep learning
• Designing Quantum Annealing Schedules using Bayesian Optimization
• Deep reinforcement learning for quantum Hamiltonian engineering
• Crowdsourcing human common sense for quantum control
• State leakage during fast decay and control of a superconducting transmon qubit
• Approximate Dynamics Lead to More Optimal Control: Efficient Exact Derivatives
• Experimental error suppression in Cross-Resonance gates via multi-derivative pulse shaping
• Beyond Games: A Systematic Review of Neural Monte Carlo Tree Search Applications
• Statistically Characterising Robustness and Fidelity of Quantum Controls and Quantum Control Algorithms
• Fast, high precision dynamics in quantum optimal control theory
• Taming quantum systems: A tutorial for using shortcuts-to-adiabaticity, quantum optimal control, and reinforcement learning
• Speeding up qubit control with bipolar single-flux-quantum pulse sequences
• Detection of entangled states supported by reinforcement learning
• Model-free optimization of power/efficiency tradeoffs in quantum thermal machines using reinforcement learning
• Reinforcement learning pulses for transmon qubit entangling gates
• Assessing Game Balance with AlphaZero: Exploring Alternative Rule Sets in Chess
• Hybrid discrete-continuous compilation of trapped-ion quantum circuits with deep reinforcement learning
• Ultrafast critical ground state preparation via bang-bang protocols
• Noise-Robust End-to-End Quantum Control using Deep Autoregressive Policy Networks
• Exploring the Optimal Cycle for Quantum Heat Engine using Reinforcement Learning
• Practical implications of SFQ-based two-qubit gates
• Classification of Equation of State in Relativistic Heavy-Ion Collisions Using Deep Learning
• Gradient projection method for constrained quantum control
• Neural network agent playing spin Hamiltonian games on a quantum computer
• Quantum optimal control in quantum technologies. Strategic report on current status, visions and goals for research in Europe
• Reinforcement Learning vs. Gradient-Based Optimisation for Robust Energy Landscape Control of Spin-1/2 Quantum Networks
• Unsupervised interpretable learning of topological indices invariant under permutations of atomic bands
• Parametrized multiqubit gate design for neutral-atom based quantum platforms
• Artificially intelligent Maxwell's demon for optimal control of open quantum systems
• Dropout is all you need: robust two-qubit gate with reinforcement learning
• Quantum Circuit Distillation and Compression
• Effect of Decoherence for Gate Operations on a Superconducting Bosonic Qubit
• The Value of Chess Squares
• Qubit Energy Tuner Based on Single Flux Quantum Circuits
• A Monte Carlo Tree Search approach to QAOA: finding a needle in the haystack
• Bayesian optimization of non-classical optomechanical correlations
• The Quantum Cartpole: A benchmark environment for non-linear reinforcement learning
• Globally optimal control of quantum dynamics
• Hybrid Reward-Driven Reinforcement Learning for Efficient Quantum Circuit Synthesis