Noise-Robust End-to-End Quantum Control using Deep Autoregressive Policy Networks

Abstract

Variational quantum eigensolvers have recently received increased attention, as they enable the use of quantum computing devices to find solutions to complex problems, such as the ground energy and ground state of strongly-correlated quantum many-body systems. In many applications, it is the optimization of both continuous and discrete parameters that poses a formidable challenge. Using reinforcement learning (RL), we present a hybrid policy gradient algorithm capable of simultaneously optimizing continuous and discrete degrees of freedom in an uncertainty-resilient way. The hybrid policy is modeled by a deep autoregressive neural network to capture causality. We employ the algorithm to prepare the ground state of the nonintegrable quantum Ising model in a unitary process, parametrized by a generalized quantum approximate optimization ansatz: the RL agent solves the discrete combinatorial problem of constructing the optimal sequences of unitaries out of a predefined set and, at the same time, it optimizes the continuous durations for which these unitaries are applied. We demonstrate the noise-robust features of the agent by considering three sources of uncertainty: classical and quantum measurement noise, and errors in the control unitary durations. Our work exhibits the beneficial synergy between reinforcement learning and quantum control.

References in corpus (66)

• Adam: A Method for Stochastic Optimization
• Supplementary information for "Quantum supremacy using a programmable superconducting processor"
• Playing Atari with Deep Reinforcement Learning
• A variational eigenvalue solver on a quantum processor
• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
• Trust Region Policy Optimization
• Machine learning and the physical sciences
• Solving the Quantum Many-Body Problem with Artificial Neural Networks
• Soft Actor-Critic Algorithms and Applications
• A Quantum Approximate Optimization Algorithm
• Machine learning phases of matter
• Shortcuts to adiabaticity: concepts, methods, and applications
• Linformer: Self-Attention with Linear Complexity
• Many-body quantum state tomography with neural networks
• Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation
• Learning phase transitions by confusion
• Discovering Phase Transitions with Unsupervised Learning
• Discovering physical concepts with neural networks
• QuSpin: a Python Package for Dynamics and Exact Diagonalisation of Quantum Many Body Systems part I: spin chains
• Reinforcement Learning with Deep Energy-Based Policies
• Chopped random-basis quantum optimization
• Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review
• QuSpin: a Python Package for Dynamics and Exact Diagonalisation of Quantum Many Body Systems. Part II: bosons, fermions and higher spins
• MADE: Masked Autoencoder for Distribution Estimation
• Reformer: The Efficient Transformer
• Minimizing irreversible losses in quantum systems by local counter-diabatic driving
• 86 PFLOPS Deep Potential Molecular Dynamics simulation of 100 million atoms with ab initio accuracy
• Flow Navigation by Smart Microswimmers via Reinforcement Learning
• Fast-forward of adiabatic dynamics in quantum mechanics
• Identifying Quantum Phase Transitions using Artificial Neural Networks on Experimental Data
• Fidelity-based Probabilistic Q-learning for Control of Quantum Systems
• Parametrized Deep Q-Networks Learning: Reinforcement Learning with Discrete-Continuous Hybrid Action Space
• Efficient variational simulation of non-trivial quantum states
• Rethinking Attention with Performers
• Unsupervised phase discovery with deep anomaly detection
• Quantum Optimization with a Novel Gibbs Objective Function and Ansatz Architecture Search
• Hybrid quantum-classical algorithms for approximate graph coloring
• Global optimization of quantum dynamics with AlphaZero deep exploration
• Branes with Brains: Exploring String Vacua with Deep Reinforcement Learning
• Reinforcement Learning for Many-Body Ground-State Preparation Inspired by Counterdiabatic Driving
• Evaluating Protein Transfer Learning with TAPE
• Minimizing estimation runtime on noisy quantum computers
• Reinforcement learning for autonomous preparation of Floquet-engineered states: Inverting the quantum Kapitza oscillator
• Measurement-based adaptation protocol with quantum reinforcement learning
• Deep Reinforcement Learning in Parameterized Action Space
• A differentiable programming method for quantum control
• Breaking Adiabatic Quantum Control with Deep Learning
• Reinforcement Learning for Digital Quantum Simulation
• Natural Evolutionary Strategies for Variational Quantum Computation
• Multi-Pass Q-Networks for Deep Reinforcement Learning with Parameterised Action Spaces
• Setting up experimental Bell test with reinforcement learning
• Discrete and Continuous Action Representation for Practical RL in Video Games
• Digital quantum simulation of molecular dynamics and control
• Policy Gradient based Quantum Approximate Optimization Algorithm
• Hessian-based optimization of constrained quantum control
• End-to-End Quantum Machine Learning Implemented with Controlled Quantum Dynamics
• Continuous-Discrete Reinforcement Learning for Hybrid Control in Robotics
• Quantifying the efficiency of state preparation via quantum variational eigensolvers
• Boosting Trust Region Policy Optimization by Normalizing Flows Policy
• Deep reinforcement learning for robust quantum optimization
• Quantum Observables for continuous control of the Quantum Approximate Optimization Algorithm via Reinforcement Learning
• Reinforcement-Learning-Based Variational Quantum Circuits Optimization for Combinatorial Problems
• Hybrid Actor-Critic Reinforcement Learning in Parameterized Action Space
• Robust Control Optimization for Quantum Approximate Optimization Algorithm
• Hierarchical Approaches for Reinforcement Learning in Parameterized Action Space
• Efficient cavity control with SNAP gates

Cited by in corpus (5)

• Noisy intermediate-scale quantum (NISQ) algorithms
• Reinforcement Learning for Many-Body Ground-State Preparation Inspired by Counterdiabatic Driving
• Quantum imaginary time evolution steered by reinforcement learning
• Hamiltonian Switching Control of Noisy Bipartite Qubit Systems
• Optimizing Quantum Variational Circuits with Deep Reinforcement Learning