Model-Free Quantum Control with Reinforcement Learning

Abstract

Model bias is an inherent limitation of the current dominant approach to optimal quantum control, which relies on a system simulation for optimization of control policies. To overcome this limitation, we propose a circuit-based approach for training a reinforcement learning agent on quantum control tasks in a model-free way. Given a continuously parameterized control circuit, the agent learns its parameters through trial-and-error interaction with the quantum system, using measurement outcomes as the only source of information about the quantum state. Focusing on control of a harmonic oscillator coupled to an ancilla qubit, we show how to reward the learning agent using measurements of experimentally available observables. We train the agent to prepare various non-classical states using both unitary control and control with adaptive measurement-based quantum feedback, and to execute logical gates on encoded qubits. This approach significantly outperforms widely used model-free methods in terms of sample efficiency. Our numerical work is of immediate relevance to superconducting circuits and trapped ions platforms where such training can be implemented in experiment, allowing complete elimination of model bias and the adaptation of quantum control policies to the specific system in which they are deployed.

References in corpus (10)

• Charge insensitive qubit design derived from the Cooper pair box
• Resolving photon number states in a superconducting circuit
• Direct Fidelity Estimation from Few Pauli Measurements
• Deep Reinforcement Learning for Quantum Gate Control
• Experimental Deep Reinforcement Learning for Error-Robust Gateset Design on a Superconducting Quantum Computer
• Gradient-based optimal control of open quantum systems using quantum trajectories and automatic differentiation
• Measurement Based Feedback Quantum Control With Deep Reinforcement Learning for Double-well Non-linear Potential
• Quantum POMDPs
• Reinforcement Learning assisted Quantum Optimization
• Quantum Observables for continuous control of the Quantum Approximate Optimization Algorithm via Reinforcement Learning

Cited by in corpus (65)

• Real-time quantum error correction beyond break-even
• Fast Universal Control of an Oscillator with Weak Dispersive Coupling to a Qubit
• Artificial Intelligence and Machine Learning for Quantum Technologies
• Learning Quantum Systems
• High-Fidelity, Frequency-Flexible Two-Qubit Fluxonium Gates with a Transmon Coupler
• Variational Quantum Reinforcement Learning via Evolutionary Optimization
• Deep Reinforcement Learning for Quantum State Preparation with Weak Nonlinear Measurements
• Physics-informed neural networks for quantum control
• Quantum Error Correction of Qudits Beyond Break-even
• Optimizing quantum gates towards the scale of logical qubits
• Realizing a deep reinforcement learning agent discovering real-time feedback control strategies for a quantum system
• Deep reinforcement learning for quantum multiparameter estimation
• Self-Correcting Quantum Many-Body Control using Reinforcement Learning with Tensor Networks
• Realization of High-Fidelity CZ Gate based on a Double-Transmon Coupler
• Gradient Ascent Pulse Engineering with Feedback
• Reinforcement learning-enhanced protocols for coherent population-transfer in three-level quantum systems
• Advances in Bosonic Quantum Error Correction with Gottesman-Kitaev-Preskill Codes: Theory, Engineering and Applications
• Deterministic remote entanglement using a chiral quantum interconnect
• Modular quantum processor with an all-to-all reconfigurable router
• Hybrid Oscillator-Qubit Quantum Processors: Instruction Set Architectures, Abstract Machine Models, and Applications
• Bidirectional multi-photon communication between remote superconducting nodes
• Variational quantum dynamics of two-dimensional rotor models
• Quantum speed limit for complex dynamics
• Engineering unsteerable quantum states with active feedback
• Taming quantum systems: A tutorial for using shortcuts-to-adiabaticity, quantum optimal control, and reinforcement learning
• Non-Markovian feedback for optimized quantum error correction
• Machine Learning-Assisted Manipulation and Readout of Molecular Spin Qubits
• Optimal entanglement generation in optomechanical systems via Krotov control of covariance matrix dynamics
• Reinforcement learning pulses for transmon qubit entangling gates
• Hybrid discrete-continuous compilation of trapped-ion quantum circuits with deep reinforcement learning
• Robust and optimal control of open quantum systems
• Experimental graybox quantum system identification and control
• Optimal control in large open quantum systems: the case of transmon readout and reset
• Exploring Quantum Control Landscape and Solution Space Complexity through Dimensionality Reduction & Optimization Algorithms
• Machine Learning for Estimation and Control of Quantum Systems
• Preparing Schrödinger cat states in a microwave cavity using a neural network
• Deep reinforcement learning for preparation of thermal and prethermal quantum states
• Quantum Similarity Testing with Convolutional Neural Networks
• TorchQC -- A framework for efficiently integrating machine and deep learning methods in quantum dynamics and control
• Entanglement engineering of optomechanical systems by reinforcement learning
• Multi-Axis Control of a Qubit in the Presence of Unknown Non-Markovian Quantum Noise
• Experimentally probing entropy reduction via iterative quantum information transfer
• Discovery of Optimal Quantum Error Correcting Codes via Reinforcement Learning
• Clifford operations and homological codes for rotors and oscillators
• Quantum optimal control in quantum technologies. Strategic report on current status, visions and goals for research in Europe
• Quantum Circuit Discovery for Fault-Tolerant Logical State Preparation with Reinforcement Learning
• Improving robustness of quantum feedback control with reinforcement learning
• Effect of Decoherence for Gate Operations on a Superconducting Bosonic Qubit
• Unitary death of Schrödinger's cat
• Fast quantum gate design with deep reinforcement learning using real-time feedback on readout signals
• Switching Time Optimization for Binary Quantum Optimal Control
• Quantum Optimal Control Theory for the Shaping of Flying Qubits
• Quantum feedback control with a transformer neural network architecture
• Optimizing measurement-based cooling by reinforcement learning
• Fast State Stabilization using Deep Reinforcement Learning for Measurement-based Quantum Feedback Control
• Error threshold in active steering protocols for few-qubit systems
• Optimising entanglement distribution policies under classical communication constraints assisted by reinforcement learning
• Generalized Parity Measurements and Efficient Large Multi-component Cat State Preparation with Quantum Signal Processing
• Quantum state-preparation control in noisy environment via most-likely paths
• Towards scalable active steering protocols for genuinely entangled state manifolds
• Hamiltonian Learning via Inverse Physics-Informed Neural Networks
• Achieving fast and robust perfect entangling gates via reinforcement learning
• Learning Feedback Mechanisms for Measurement-Based Variational Quantum State Preparation
• Feedback cooling of fermionic atoms in optical lattices
• Diverse efficiency of observable optimization for four-level quantum systems with higher-order traps