Policy Gradient based Quantum Approximate Optimization Algorithm

Abstract

The quantum approximate optimization algorithm (QAOA), as a hybrid quantum/classical algorithm, has received much interest recently. QAOA can also be viewed as a variational ansatz for quantum control. However, its direct application to emergent quantum technology encounters additional physical constraints: (i) the states of the quantum system are not observable; (ii) obtaining the derivatives of the objective function can be computationally expensive or even inaccessible in experiments, and (iii) the values of the objective function may be sensitive to various sources of uncertainty, as is the case for noisy intermediate-scale quantum (NISQ) devices. Taking such constraints into account, we show that policy-gradient-based reinforcement learning (RL) algorithms are well suited for optimizing the variational parameters of QAOA in a noise-robust fashion, opening up the way for developing RL techniques for continuous quantum control. This is advantageous to help mitigate and monitor the potentially unknown sources of errors in modern quantum simulators. We analyze the performance of the algorithm for quantum state transfer problems in single- and multi-qubit systems, subject to various sources of noise such as error terms in the Hamiltonian, or quantum uncertainty in the measurement process. We show that, in noisy setups, it is capable of outperforming state-of-the-art existing optimization algorithms.

Mathematical and Scientific Machine Learning Conference (MSML) 2020

References in corpus (29)

• Adam: A Method for Stochastic Optimization
• A variational eigenvalue solver on a quantum processor
• Trust Region Policy Optimization
• QuTiP 2: A Python framework for the dynamics of open quantum systems
• Barren plateaus in quantum neural network training landscapes
• QuTiP: An open-source Python framework for the dynamics of open quantum systems
• A Quantum Approximate Optimization Algorithm
• Many-body quantum state tomography with neural networks
• From the Quantum Approximate Optimization Algorithm to a Quantum Alternating Operator Ansatz
• Density estimation using Real NVP
• QuSpin: a Python Package for Dynamics and Exact Diagonalisation of Quantum Many Body Systems part I: spin chains
• QuSpin: a Python Package for Dynamics and Exact Diagonalisation of Quantum Many Body Systems. Part II: bosons, fermions and higher spins
• MADE: Masked Autoencoder for Distribution Estimation
• Fidelity-based Probabilistic Q-learning for Control of Quantum Systems
• Efficient variational simulation of non-trivial quantum states
• Deep Reinforcement Learning for Quantum Gate Control
• Learning Robust and High-Precision Quantum Controls
• Robust control of quantum gates via sequential convex programming
• Global optimization of quantum dynamics with AlphaZero deep exploration
• Quantum approximate optimization is computationally universal
• Reinforcement learning for autonomous preparation of Floquet-engineered states: Inverting the quantum Kapitza oscillator
• Glassy Phase of Optimal Quantum Control
• A differentiable programming method for quantum control
• Broken symmetry in a two-qubit quantum control landscape
• Deep reinforcement learning for robust quantum optimization
• Optimizing QAOA: Success Probability and Runtime Dependence on Circuit Depth
• A Quantum Approximate Optimization Algorithm for continuous problems
• Robust Control Optimization for Quantum Approximate Optimization Algorithm
• Manipulation of Spin Dynamics by Deep Reinforcement Learning Agent

Cited by in corpus (17)

• Hybrid quantum-classical algorithms for approximate graph coloring
• A comprehensive review of Quantum Machine Learning: from NISQ to Fault Tolerance
• Reinforcement Learning for Many-Body Ground-State Preparation Inspired by Counterdiabatic Driving
• A reinforcement learning approach to rare trajectory sampling
• Reinforcement Learning for Digital Quantum Simulation
• Digital quantum simulation of molecular dynamics and control
• Quantum Architecture Search via Deep Reinforcement Learning
• Reinforcement learning of rare diffusive dynamics
• Predicting quantum dynamical cost landscapes with deep learning
• Automatic Depth Optimization for Quantum Approximate Optimization Algorithm
• Robust optimization for quantum reinforcement learning control using partial observations
• Noise-Robust End-to-End Quantum Control using Deep Autoregressive Policy Networks
• Parameter Setting Heuristics Make the Quantum Approximate Optimization Algorithm Suitable for the Early Fault-Tolerant Era
• Hamiltonian Switching Control of Noisy Bipartite Qubit Systems
• Policy Gradient Approach to Compilation of Variational Quantum Circuits
• Optimizing Quantum Variational Circuits with Deep Reinforcement Learning
• Topological and geometric patterns in optimal bang-bang protocols for variational quantum algorithms: application to the $XXZ$ model on the square lattice