Noise-Robust End-to-End Quantum Control using Deep Autoregressive Policy Networks

Abstract

Variational quantum eigensolvers have recently received increased attention, as they enable the use of quantum computing devices to find solutions to complex problems, such as the ground energy and ground state of strongly-correlated quantum many-body systems. In many applications, it is the optimization of both continuous and discrete parameters that poses a formidable challenge. Using reinforcement learning (RL), we present a hybrid policy gradient algorithm capable of simultaneously optimizing continuous and discrete degrees of freedom in an uncertainty-resilient way. The hybrid policy is modeled by a deep autoregressive neural network to capture causality. We employ the algorithm to prepare the ground state of the nonintegrable quantum Ising model in a unitary process, parametrized by a generalized quantum approximate optimization ansatz: the RL agent solves the discrete combinatorial problem of constructing the optimal sequences of unitaries out of a predefined set and, at the same time, it optimizes the continuous durations for which these unitaries are applied. We demonstrate the noise-robust features of the agent by considering three sources of uncertainty: classical and quantum measurement noise, and errors in the control unitary durations. Our work exhibits the beneficial synergy between reinforcement learning and quantum control.

References in corpus (21)

• Supplementary information for "Quantum supremacy using a programmable superconducting processor"
• A Quantum Approximate Optimization Algorithm
• Linformer: Self-Attention with Linear Complexity
• Learning phase transitions by confusion
• QuSpin: a Python Package for Dynamics and Exact Diagonalisation of Quantum Many Body Systems part I: spin chains
• MADE: Masked Autoencoder for Distribution Estimation
• Reformer: The Efficient Transformer
• Rethinking Attention with Performers
• Hybrid quantum-classical algorithms for approximate graph coloring
• Branes with Brains: Exploring String Vacua with Deep Reinforcement Learning
• Reinforcement Learning for Many-Body Ground-State Preparation Inspired by Counterdiabatic Driving
• Evaluating Protein Transfer Learning with TAPE
• Breaking Adiabatic Quantum Control with Deep Learning
• Natural Evolutionary Strategies for Variational Quantum Computation
• Multi-Pass Q-Networks for Deep Reinforcement Learning with Parameterised Action Spaces
• Discrete and Continuous Action Representation for Practical RL in Video Games
• Continuous-Discrete Reinforcement Learning for Hybrid Control in Robotics
• Quantum Observables for continuous control of the Quantum Approximate Optimization Algorithm via Reinforcement Learning
• Hybrid Actor-Critic Reinforcement Learning in Parameterized Action Space
• Robust Control Optimization for Quantum Approximate Optimization Algorithm
• Efficient cavity control with SNAP gates

Cited by in corpus (5)

• Noisy intermediate-scale quantum (NISQ) algorithms
• Reinforcement Learning for Many-Body Ground-State Preparation Inspired by Counterdiabatic Driving
• Quantum imaginary time evolution steered by reinforcement learning
• Hamiltonian Switching Control of Noisy Bipartite Qubit Systems
• Optimizing Quantum Variational Circuits with Deep Reinforcement Learning