Model-Free Quantum Control with Reinforcement Learning

Abstract

Model bias is an inherent limitation of the current dominant approach to optimal quantum control, which relies on a system simulation for optimization of control policies. To overcome this limitation, we propose a circuit-based approach for training a reinforcement learning agent on quantum control tasks in a model-free way. Given a continuously parameterized control circuit, the agent learns its parameters through trial-and-error interaction with the quantum system, using measurement outcomes as the only source of information about the quantum state. Focusing on control of a harmonic oscillator coupled to an ancilla qubit, we show how to reward the learning agent using measurements of experimentally available observables. We train the agent to prepare various non-classical states using both unitary control and control with adaptive measurement-based quantum feedback, and to execute logical gates on encoded qubits. This approach significantly outperforms widely used model-free methods in terms of sample efficiency. Our numerical work is of immediate relevance to superconducting circuits and trapped ions platforms where such training can be implemented in experiment, allowing complete elimination of model bias and the adaptation of quantum control policies to the specific system in which they are deployed.

References in corpus (71)

• SciPy 1.0--Fundamental Algorithms for Scientific Computing in Python
• LSTM: A Search Space Odyssey
• Cavity Optomechanics
• Charge insensitive qubit design derived from the Cooper pair box
• Variational Quantum Algorithms
• Cavity quantum electrodynamics for superconducting electrical circuits: an architecture for quantum computation
• Circuit Quantum Electrodynamics
• A Quantum Engineer's Guide to Superconducting Qubits
• Encoding a qubit in an oscillator
• Trapped-Ion Quantum Computing: Progress and Challenges
• An Introduction to Deep Reinforcement Learning
• Resolving photon number states in a superconducting circuit
• Negativity of the Wigner function as an indicator of nonclassicality
• A simple formula for the average gate fidelity of a quantum dynamical operation
• Real-time quantum feedback prepares and stabilizes photon number states
• Direct Fidelity Estimation from Few Pauli Measurements
• Quantum error correction of a qubit encoded in grid states of an oscillator
• New class of quantum error-correcting codes for a bosonic mode
• Chopped random-basis quantum optimization
• Encoding a qubit in a trapped-ion mechanical oscillator
• Practical characterization of quantum devices without tomography
• Implementing a Universal Gate Set on a Logical Qubit Encoded in an Oscillator
• Reinforcement Learning in Different Phases of Quantum Control
• Second order gradient ascent pulse engineering
• Demonstration of quantum error correction and universal gate set on a binomial bosonic logical qubit
• Measuring and Suppressing Quantum State Leakage in a Superconducting Qubit
• Optimal quantum control using randomized benchmarking
• Reinforcement Learning with Neural Networks for Quantum Feedback
• Cavity State Manipulation Using Photon-Number Selective Phase Gates
• Universal Control of an Oscillator with Dispersive Coupling to a Qubit
• Leakage reduction in fast superconducting qubit gates via optimal control
• Fidelity-based Probabilistic Q-learning for Control of Quantum Systems
• Optimizing Quantum Error Correction Codes with Reinforcement Learning
• Theory of quantum system certification: a tutorial
• Schrodinger's catapult: Launching multiphoton quantum states from a microwave cavity memory
• Single-Mode Displacement Sensor
• Speedup for quantum optimal control from automatic differentiation based on graphics processing units
• When does reinforcement learning stand out in quantum control? A comparative study on state preparation
• Encoding a Qubit into a Cavity Mode in Circuit-QED using Phase Estimation
• Deep Reinforcement Learning for Quantum Gate Control
• Experimental Deep Reinforcement Learning for Error-Robust Gateset Design on a Superconducting Quantum Computer
• Generalizable control for quantum parameter estimation through reinforcement learning
• Quantum error correction for the toric code using deep reinforcement learning
• Stabilization of Finite-Energy Gottesman-Kitaev-Preskill States
• Using a Recurrent Neural Network to Reconstruct Quantum Dynamics of a Superconducting Qubit from Physical Observations
• Encoding an oscillator into many oscillators
• Time-domain characterization and correction of on-chip distortion of control pulses in a quantum processor
• Gradient-based optimal control of open quantum systems using quantum trajectories and automatic differentiation
• Error-corrected gates on an encoded qubit
• Experimental Phase Estimation Enhanced By Machine Learning
• Coherent Transport of Quantum States by Deep Reinforcement Learning
• Global optimization of quantum dynamics with AlphaZero deep exploration
• Measurement Based Feedback Quantum Control With Deep Reinforcement Learning for Double-well Non-linear Potential
• Direct characteristic-function tomography of quantum states of the trapped-ion motional oscillator
• Restless Tuneup of High-Fidelity Qubit Gates
• Heisenberg limited single-mode quantum metrology
• Quantum POMDPs
• Modelling Non-Markovian Quantum Processes with Recurrent Neural Networks
• Reinforcement Learning assisted Quantum Optimization
• Reinforcement learning for autonomous preparation of Floquet-engineered states: Inverting the quantum Kapitza oscillator
• Deep Reinforcement Learning Control of Quantum Cartpoles
• Quantum Channel Construction with Circuit Quantum Electrodynamics
• Quantum optimal control of multi-level dissipative quantum systems with Reinforcement Learning
• Path-Independent Quantum Gates with Noisy Ancilla
• Improving the dynamics of quantum sensors with reinforcement learning
• Classifying global state preparation via deep reinforcement learning
• Reinforcement learning for optimal error correction of toric codes
• In situ characterization of qubit control lines: a qubit as a vector network analyzer
• Stochastic Estimation of Dynamical Variables
• Real-time calibration of coherent-state receivers: learning by trial and error
• Quantum Observables for continuous control of the Quantum Approximate Optimization Algorithm via Reinforcement Learning

Cited by in corpus (67)

• Real-time quantum error correction beyond break-even
• Fast Universal Control of an Oscillator with Weak Dispersive Coupling to a Qubit
• Artificial Intelligence and Machine Learning for Quantum Technologies
• Learning Quantum Systems
• High-Fidelity, Frequency-Flexible Two-Qubit Fluxonium Gates with a Transmon Coupler
• Variational Quantum Reinforcement Learning via Evolutionary Optimization
• Deep Reinforcement Learning for Quantum State Preparation with Weak Nonlinear Measurements
• Physics-informed neural networks for quantum control
• Quantum Error Correction of Qudits Beyond Break-even
• Optimizing quantum gates towards the scale of logical qubits
• Realizing a deep reinforcement learning agent discovering real-time feedback control strategies for a quantum system
• Deep reinforcement learning for quantum multiparameter estimation
• Self-Correcting Quantum Many-Body Control using Reinforcement Learning with Tensor Networks
• Gradient Ascent Pulse Engineering with Feedback
• Realization of High-Fidelity CZ Gate based on a Double-Transmon Coupler
• Reinforcement learning-enhanced protocols for coherent population-transfer in three-level quantum systems
• Advances in Bosonic Quantum Error Correction with Gottesman-Kitaev-Preskill Codes: Theory, Engineering and Applications
• Deterministic remote entanglement using a chiral quantum interconnect
• Modular quantum processor with an all-to-all reconfigurable router
• Hybrid Oscillator-Qubit Quantum Processors: Instruction Set Architectures, Abstract Machine Models, and Applications
• Bidirectional multi-photon communication between remote superconducting nodes
• Variational quantum dynamics of two-dimensional rotor models
• Quantum speed limit for complex dynamics
• Engineering unsteerable quantum states with active feedback
• Taming quantum systems: A tutorial for using shortcuts-to-adiabaticity, quantum optimal control, and reinforcement learning
• Quantum Logic Gate Synthesis as a Markov Decision Process
• Non-Markovian feedback for optimized quantum error correction
• Machine Learning-Assisted Manipulation and Readout of Molecular Spin Qubits
• Optimal entanglement generation in optomechanical systems via Krotov control of covariance matrix dynamics
• Reinforcement learning pulses for transmon qubit entangling gates
• Hybrid discrete-continuous compilation of trapped-ion quantum circuits with deep reinforcement learning
• Experimental graybox quantum system identification and control
• Optimal control in large open quantum systems: the case of transmon readout and reset
• Robust and optimal control of open quantum systems
• Machine Learning for Estimation and Control of Quantum Systems
• Exploring Quantum Control Landscape and Solution Space Complexity through Dimensionality Reduction & Optimization Algorithms
• Preparing Schrödinger cat states in a microwave cavity using a neural network
• Deep reinforcement learning for preparation of thermal and prethermal quantum states
• Quantum Similarity Testing with Convolutional Neural Networks
• TorchQC -- A framework for efficiently integrating machine and deep learning methods in quantum dynamics and control
• Entanglement engineering of optomechanical systems by reinforcement learning
• Multi-Axis Control of a Qubit in the Presence of Unknown Non-Markovian Quantum Noise
• Clifford operations and homological codes for rotors and oscillators
• Discovery of Optimal Quantum Error Correcting Codes via Reinforcement Learning
• Experimentally probing entropy reduction via iterative quantum information transfer
• Quantum optimal control in quantum technologies. Strategic report on current status, visions and goals for research in Europe
• Improving robustness of quantum feedback control with reinforcement learning
• Quantum Circuit Discovery for Fault-Tolerant Logical State Preparation with Reinforcement Learning
• Minimal Time Robust Control for Two Superconducting Qubits
• Effect of Decoherence for Gate Operations on a Superconducting Bosonic Qubit
• Unitary death of Schrödinger's cat
• Fast quantum gate design with deep reinforcement learning using real-time feedback on readout signals
• Quantum Optimal Control Theory for the Shaping of Flying Qubits
• Quantum feedback control with a transformer neural network architecture
• Switching Time Optimization for Binary Quantum Optimal Control
• Fast State Stabilization using Deep Reinforcement Learning for Measurement-based Quantum Feedback Control
• Error threshold in active steering protocols for few-qubit systems
• Optimizing measurement-based cooling by reinforcement learning
• Quantum state-preparation control in noisy environment via most-likely paths
• Towards scalable active steering protocols for genuinely entangled state manifolds
• Optimising entanglement distribution policies under classical communication constraints assisted by reinforcement learning
• Hamiltonian Learning via Inverse Physics-Informed Neural Networks
• Achieving fast and robust perfect entangling gates via reinforcement learning
• Generalized Parity Measurements and Efficient Large Multi-component Cat State Preparation with Quantum Signal Processing
• Feedback cooling of fermionic atoms in optical lattices
• Diverse efficiency of observable optimization for four-level quantum systems with higher-order traps
• Learning Feedback Mechanisms for Measurement-Based Variational Quantum State Preparation