Realizing a deep reinforcement learning agent discovering real-time feedback control strategies for a quantum system

Abstract

To realize the full potential of quantum technologies, finding good strategies to control quantum information processing devices in real time becomes increasingly important. Usually these strategies require a precise understanding of the device itself, which is generally not available. Model-free reinforcement learning circumvents this need by discovering control strategies from scratch without relying on an accurate description of the quantum system. Furthermore, important tasks like state preparation, gate teleportation and error correction need feedback at time scales much shorter than the coherence time, which for superconducting circuits is in the microsecond range. Developing and training a deep reinforcement learning agent able to operate in this real-time feedback regime has been an open challenge. Here, we have implemented such an agent in the form of a latency-optimized deep neural network on a field-programmable gate array (FPGA). We demonstrate its use to efficiently initialize a superconducting qubit into a target state. To train the agent, we use model-free reinforcement learning that is based solely on measurement data. We study the agent's performance for strong and weak measurements, and for three-level readout, and compare with simple strategies based on thresholding. This demonstration motivates further research towards adoption of reinforcement learning for real-time feedback control of quantum devices and more generally any physical system requiring learnable low-latency feedback control.

arXiv comment14 pages, 10 figures

References in corpus (30)

• Cavity quantum electrodynamics for superconducting electrical circuits: an architecture for quantum computation
• Machine learning and the physical sciences
• Circuit Quantum Electrodynamics
• Approaching Unit Visibility for Control of a Superconducting Qubit with Dispersive Readout
• Real-time quantum error correction beyond break-even
• Realizing Rapid, High-Fidelity, Single-Shot Dispersive Readout of Superconducting Qubits
• Realization of Deterministic Quantum Teleportation with Solid State Qubits
• Deterministic teleportation of a quantum gate between two logical qubits
• Quantum feedback: theory, experiments, and applications
• Fast and Unconditional All-Microwave Reset of a Superconducting Qubit
• Feedback control of a solid-state qubit using high-fidelity projective measurement
• Protocols for optimal readout of qubits using a continuous quantum nondemolition measurement
• Artificial Intelligence and Machine Learning for Quantum Technologies
• Dynamics of simultaneously measured non-commuting observables
• Fidelity-based Probabilistic Q-learning for Control of Quantum Systems
• Optimizing Quantum Error Correction Codes with Reinforcement Learning
• Repeated multi-qubit readout and feedback with a mixed-species trapped-ion register
• Machine learning for discriminating quantum measurement trajectories and improving readout
• Model-Free Quantum Control with Reinforcement Learning
• Experimental Deep Reinforcement Learning for Error-Robust Gateset Design on a Superconducting Quantum Computer
• Persistent control of a superconducting qubit by stroboscopic measurement feedback
• Topological Quantum Compiling with Reinforcement Learning
• Entanglement Stabilization using Parity Detection and Real-Time Feedback in Superconducting Circuits
• Measurement Based Feedback Quantum Control With Deep Reinforcement Learning for Double-well Non-linear Potential
• Deep Reinforcement Learning for Quantum State Preparation with Weak Nonlinear Measurements
• Reinforcement Learning Decoders for Fault-Tolerant Quantum Computation
• Low-Latency Digital Signal Processing for Feedback and Feedforward in Quantum Computing and Communication
• Faster State Preparation across Quantum Phase Transition Assisted by Reinforcement Learning
• Deep Reinforcement Learning for Efficient Measurement of Quantum Devices
• Deep reinforcement learning for quantum Hamiltonian engineering

Cited by in corpus (23)

• Single chip photonic deep neural network with accelerated training
• Artificial Intelligence for Quantum Computing
• Taming quantum systems: A tutorial for using shortcuts-to-adiabaticity, quantum optimal control, and reinforcement learning
• Non-Markovian feedback for optimized quantum error correction
• Machine Learning for Estimation and Control of Quantum Systems
• Preparing Schrödinger cat states in a microwave cavity using a neural network
• Optimizing ZX-Diagrams with Deep Reinforcement Learning
• Designing fast quantum gates using optimal control with a reinforcement-learning ansatz
• Efficient Learning for Linear Properties of Bounded-Gate Quantum Circuits
• Entanglement engineering of optomechanical systems by reinforcement learning
• Improving robustness of quantum feedback control with reinforcement learning
• Realization of Constant-Depth Fan-Out with Real-Time Feedforward on a Superconducting Quantum Processor
• Reinforcement learning for hardware acceleration and system-level optimization on FPGA and integrated-circuit platforms: A survey
• Real-time adaptive tracking of fluctuating relaxation rates in superconducting qubits
• Artificially intelligent Maxwell's demon for optimal control of open quantum systems
• Quantum feedback control with a transformer neural network architecture
• Efficient Qubit Calibration by Binary-Search Hamiltonian Tracking
• No-Collapse Accurate Quantum Feedback Control via Conditional State Tomography
• Superconducting Qubit Readout Using Next-Generation Reservoir Computing
• Achieving fast and robust perfect entangling gates via reinforcement learning
• CDJ-Pontryagin Optimal Control for General Continuously Monitored Quantum Systems
• Feedback cooling of fermionic atoms in optical lattices
• Low-latency machine learning FPGA accelerator for multi-qubit-state discrimination