Reinforcement Learning for Many-Body Ground-State Preparation Inspired by Counterdiabatic Driving

Abstract

The quantum alternating operator ansatz (QAOA) is a prominent example of variational quantum algorithms. We propose a generalized QAOA called CD-QAOA, which is inspired by the counterdiabatic driving procedure, designed for quantum many-body systems and optimized using a reinforcement learning (RL) approach. The resulting hybrid control algorithm proves versatile in preparing the ground state of quantum-chaotic many-body spin chains by minimizing the energy. We show that using terms occurring in the adiabatic gauge potential as generators of additional control unitaries, it is possible to achieve fast high-fidelity many-body control away from the adiabatic regime. While each unitary retains the conventional QAOA-intrinsic continuous control degree of freedom such as the time duration, we consider the order of the multiple available unitaries appearing in the control sequence as an additional discrete optimization problem. Endowing the policy gradient algorithm with an autoregressive deep learning architecture to capture causality, we train the RL agent to construct optimal sequences of unitaries. The algorithm has no access to the quantum state, and we find that the protocol learned on small systems may generalize to larger systems. By scanning a range of protocol durations, we present numerical evidence for a finite quantum speed limit in the nonintegrable mixed-field spin-1/2 Ising and Lipkin-Meshkov-Glick models, and for the suitability to prepare ground states of the spin-1 Heisenberg chain in the long-range and topologically ordered parameter regimes. This work paves the way to incorporate recent success from deep learning for the purpose of quantum many-body control.

References in corpus (65)

• Many-Body Physics with Ultracold Gases
• Supplementary information for "Quantum supremacy using a programmable superconducting processor"
• A variational eigenvalue solver on a quantum processor
• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
• The nitrogen-vacancy colour centre in diamond
• Machine learning and the physical sciences
• Barren plateaus in quantum neural network training landscapes
• Ultracold atomic gases in optical lattices: mimicking condensed matter physics and beyond
• Hybrid quantum circuits: Superconducting circuits interacting with other quantum systems
• Entanglement spectrum of a topological phase in one dimension
• Quantum computing with trapped ions
• Cost Function Dependent Barren Plateaus in Shallow Parametrized Quantum Circuits
• A high-bias, low-variance introduction to Machine Learning for physicists
• From the Quantum Approximate Optimization Algorithm to a Quantum Alternating Operator Ansatz
• Quantum Approximate Optimization Algorithm: Performance, Mechanism, and Implementation on Near-Term Devices
• Probing condensed matter physics with magnetometry based on nitrogen-vacancy centres in diamond
• Fisher Information and entanglement of non-Gaussian spin states
• QuSpin: a Python Package for Dynamics and Exact Diagonalisation of Quantum Many Body Systems part I: spin chains
• Chopped random-basis quantum optimization
• An initialization strategy for addressing barren plateaus in parametrized quantum circuits
• QuSpin: a Python Package for Dynamics and Exact Diagonalisation of Quantum Many Body Systems. Part II: bosons, fermions and higher spins
• Geometry and non-adiabatic response in quantum and classical systems
• Reinforcement Learning in Different Phases of Quantum Control
• Minimizing irreversible losses in quantum systems by local counter-diabatic driving
• Deep autoregressive models for the efficient variational simulation of many-body quantum systems
• Ground State Phase Diagram of S=1 XXZ Chains with Uniaxial Single-Ion-Type Anisotropy
• Reinforcement Learning with Neural Networks for Quantum Feedback
• Solving Statistical Mechanics Using Variational Autoregressive Networks
• Optimizing Quantum Error Correction Codes with Reinforcement Learning
• Adiabatic eigenstate deformations as a sensitive probe for quantum chaos
• Machine Learning for Quantum Matter
• Efficient variational simulation of non-trivial quantum states
• Shortcuts to Adiabaticity in Digitized Adiabatic Quantum Computing
• Counterdiabaticity and the quantum approximate optimization algorithm
• Quantum Optimization with a Novel Gibbs Objective Function and Ansatz Architecture Search
• Geometric Speed Limit of Accessible Many-Body State Preparation
• Coherent Transport of Quantum States by Deep Reinforcement Learning
• Global optimization of quantum dynamics with AlphaZero deep exploration
• Improving the Performance of Deep Quantum Optimization Algorithms with Continuous Gate Sets
• Reinforcement Learning assisted Quantum Optimization
• Reinforcement learning for autonomous preparation of Floquet-engineered states: Inverting the quantum Kapitza oscillator
• Glassy Phase of Optimal Quantum Control
• Using models to improve optimizers for variational quantum algorithms
• Measurement-based adaptation protocol with quantum reinforcement learning
• Characterizing the loss landscape of variational quantum circuits
• Protecting Spin Coherence in a Tunable Heisenberg Model
• Rapid counter-diabatic sweeps in lattice gauge adiabatic quantum computing
• A differentiable programming method for quantum control
• A reinforcement learning approach to rare trajectory sampling
• Breaking Adiabatic Quantum Control with Deep Learning
• Reinforcement Learning for Digital Quantum Simulation
• Making Trotters Sprint: A Variational Imaginary Time Ansatz for Quantum Many-body Systems
• Ground-state fidelity of the spin-1 Heisenberg chain with single ion anisotropy: quantum renormalization group and exact diagonalization approaches
• Optimal quantum control with poor statistics
• Policy Gradient based Quantum Approximate Optimization Algorithm
• End-to-End Quantum Machine Learning Implemented with Controlled Quantum Dynamics
• Broken symmetry in a two-qubit quantum control landscape
• Quantifying the efficiency of state preparation via quantum variational eigensolvers
• Variational Schrieffer-Wolff Transformations for Quantum Many-Body Dynamics
• Shortcuts to adiabaticity in the infinite-range Ising model by mean-field counter-diabatic driving
• Work statistics and symmetry breaking in an excited state quantum phase transition
• Quantum Observables for continuous control of the Quantum Approximate Optimization Algorithm via Reinforcement Learning
• Reinforcement-Learning-Based Variational Quantum Circuits Optimization for Combinatorial Problems
• Noise-Robust End-to-End Quantum Control using Deep Autoregressive Policy Networks
• Efficient cavity control with SNAP gates

Cited by in corpus (55)

• Quantum circuit architecture search for variational quantum algorithms
• Artificial Intelligence and Machine Learning for Quantum Technologies
• Digitized-counterdiabatic quantum approximate optimization algorithm
• Counterdiabatic Optimised Local Driving
• Digitized-Counterdiabatic Quantum Algorithm for Protein Folding
• Portfolio Optimization with Digitized-Counterdiabatic Quantum Algorithms
• Approximate Autonomous Quantum Error Correction with Reinforcement Learning
• Shortcuts to Adiabaticity in Krylov Space
• Digitized-Counterdiabatic Quantum Optimization
• Quantum optimization via four-body Rydberg gates
• Gradient Ascent Pulse Engineering with Feedback
• Parameter Setting in Quantum Approximate Optimization of Weighted Problems
• Quantum imaginary time evolution steered by reinforcement learning
• Shortcuts to adiabaticity: theoretical framework, relations between different methods, and versatile approximations
• Reinforcement Learning Assisted Recursive QAOA
• Variational counterdiabatic driving of the Hubbard model for ground-state preparation
• Efficient Online Quantum Generative Adversarial Learning Algorithms with Applications
• Efficient Bipartite Entanglement Detection Scheme with a Quantum Adversarial Solver
• Optimizing Counterdiabaticity by Variational Quantum Circuits
• Solution of SAT Problems with the Adaptive-Bias Quantum Approximate Optimization Algorithm
• Quantum Optimization for Training Quantum Neural Networks
• Taming quantum systems: A tutorial for using shortcuts-to-adiabaticity, quantum optimal control, and reinforcement learning
• Counterdiabatic optimized driving in quantum phase sensitive models
• Detection of entangled states supported by reinforcement learning
• Continuous quantum gate sets and pulse class meta-optimization
• Hybrid discrete-continuous compilation of trapped-ion quantum circuits with deep reinforcement learning
• Convergence of Digitized-Counterdiabatic QAOA: circuit depth versus free parameters
• Counterdiabatic Formalism of Shortcuts to Adiabaticity
• Neural networks in quantum many-body physics: a hands-on tutorial
• Deep reinforcement learning for preparation of thermal and prethermal quantum states
• Fast generation of entanglement between coupled spins using optimization and deep learning methods
• Non-Adiabatic Quantum Optimization for Crossing Quantum Phase Transitions
• Random coordinate descent: a simple alternative for optimizing parameterized quantum circuits
• Exponentially improved efficient machine learning for quantum many-body states with provable guarantees
• Improving the performance of quantum approximate optimization for preparing non-trivial quantum states without translational symmetry
• Noise-Robust End-to-End Quantum Control using Deep Autoregressive Policy Networks
• Entanglement engineering of optomechanical systems by reinforcement learning
• Efficient DCQO Algorithm within the Impulse Regime for Portfolio Optimization
• A numerical approach for calculating exact non-adiabatic terms in quantum dynamics
• Benchmarking hybrid digitized-counterdiabatic quantum optimization
• Deterministic generation of nonclassical mechanical states in cavity optomechanics via reinforcement learning
• Characterization of a driven two-level quantum system by Supervised Learning
• Counterdiabatic driving for long-lived singlet state preparation
• Dropout is all you need: robust two-qubit gate with reinforcement learning
• Controlling quantum many-body systems using reduced-order modelling
• A Monte Carlo Tree Search approach to QAOA: finding a needle in the haystack
• Shortcuts to adiabaticity in open quantum critical systems
• Policy Gradient Approach to Compilation of Variational Quantum Circuits
• Fighting Exponentially Small Gaps by Counterdiabatic Driving
• Fast State Stabilization using Deep Reinforcement Learning for Measurement-based Quantum Feedback Control
• Quantum control without quantum states
• Optimizing measurement-based cooling by reinforcement learning
• Symmetry-enhanced Counterdiabatic Quantum Algorithm for Qudits
• Towards neural reinforcement learning for large deviations in nonequilibrium systems with memory
• Improving Variational Counterdiabatic Driving with Weighted Actions and Computer Algebra