Realizing a deep reinforcement learning agent discovering real-time feedback control strategies for a quantum system

Abstract

To realize the full potential of quantum technologies, finding good strategies to control quantum information processing devices in real time becomes increasingly important. Usually these strategies require a precise understanding of the device itself, which is generally not available. Model-free reinforcement learning circumvents this need by discovering control strategies from scratch without relying on an accurate description of the quantum system. Furthermore, important tasks like state preparation, gate teleportation and error correction need feedback at time scales much shorter than the coherence time, which for superconducting circuits is in the microsecond range. Developing and training a deep reinforcement learning agent able to operate in this real-time feedback regime has been an open challenge. Here, we have implemented such an agent in the form of a latency-optimized deep neural network on a field-programmable gate array (FPGA). We demonstrate its use to efficiently initialize a superconducting qubit into a target state. To train the agent, we use model-free reinforcement learning that is based solely on measurement data. We study the agent's performance for strong and weak measurements, and for three-level readout, and compare with simple strategies based on thresholding. This demonstration motivates further research towards adoption of reinforcement learning for real-time feedback control of quantum devices and more generally any physical system requiring learnable low-latency feedback control.

14 pages, 10 figures

References in corpus (8)

• Approaching Unit Visibility for Control of a Superconducting Qubit with Dispersive Readout
• Real-time quantum error correction beyond break-even
• Feedback control of a solid-state qubit using high-fidelity projective measurement
• Protocols for optimal readout of qubits using a continuous quantum nondemolition measurement
• Artificial Intelligence and Machine Learning for Quantum Technologies
• Dynamics of simultaneously measured non-commuting observables
• Machine learning for discriminating quantum measurement trajectories and improving readout
• Faster State Preparation across Quantum Phase Transition Assisted by Reinforcement Learning

Cited by in corpus (22)

• Single chip photonic deep neural network with accelerated training
• Artificial Intelligence for Quantum Computing
• Taming quantum systems: A tutorial for using shortcuts-to-adiabaticity, quantum optimal control, and reinforcement learning
• Non-Markovian feedback for optimized quantum error correction
• Machine Learning for Estimation and Control of Quantum Systems
• Preparing Schrödinger cat states in a microwave cavity using a neural network
• Optimizing ZX-Diagrams with Deep Reinforcement Learning
• Efficient Learning for Linear Properties of Bounded-Gate Quantum Circuits
• Entanglement engineering of optomechanical systems by reinforcement learning
• Designing fast quantum gates using optimal control with a reinforcement-learning ansatz
• Realization of Constant-Depth Fan-Out with Real-Time Feedforward on a Superconducting Quantum Processor
• Improving robustness of quantum feedback control with reinforcement learning
• Artificially intelligent Maxwell's demon for optimal control of open quantum systems
• Real-time adaptive tracking of fluctuating relaxation rates in superconducting qubits
• Efficient Qubit Calibration by Binary-Search Hamiltonian Tracking
• Quantum feedback control with a transformer neural network architecture
• Superconducting Qubit Readout Using Next-Generation Reservoir Computing
• No-Collapse Accurate Quantum Feedback Control via Conditional State Tomography
• Achieving fast and robust perfect entangling gates via reinforcement learning
• Low-latency machine learning FPGA accelerator for multi-qubit-state discrimination
• CDJ-Pontryagin Optimal Control for General Continuously Monitored Quantum Systems
• Feedback cooling of fermionic atoms in optical lattices