Quantum error correction for the toric code using deep reinforcement learning

Abstract

We implement a quantum error correction algorithm for bit-flip errors on the topological toric code using deep reinforcement learning. An action-value Q-function encodes the discounted value of moving a defect to a neighboring site on the square grid (the action) depending on the full set of defects on the torus (the syndrome or state). The Q-function is represented by a deep convolutional neural network. Using the translational invariance on the torus allows for viewing each defect from a central perspective which significantly simplifies the state space representation independently of the number of defect pairs. The training is done using experience replay, where data from the algorithm being played out is stored and used for mini-batch upgrade of the Q-network. We find performance which is close to, and for small error rates asymptotically equivalent to, that achieved by the Minimum Weight Perfect Matching algorithm for code distances up to $d=7$. Our results show that it is possible for a self-trained agent without supervision or support algorithms to find a decoding scheme that performs on par with hand-made algorithms, opening up for future machine engineered decoders for more general error models and error correcting codes.

V3. As published

References in corpus (12)

• Surface codes: Towards practical large-scale quantum computation
• Learning phase transitions by confusion
• Topological fault-tolerance in cluster state quantum computation
• Reinforcement Learning with Neural Networks for Quantum Feedback
• Efficient Algorithms for Maximum Likelihood Decoding in the Surface Code
• Ultrahigh Error Threshold for Surface Codes with Biased Noise
• Machine learning for many-body physics: The case of the Anderson impurity model
• Neural Belief-Propagation Decoders for Quantum Error-Correcting Codes
• Deep neural decoders for near term fault-tolerant experiments
• Advantages of versatile neural-network decoding for topological codes
• Cellular-automaton decoders with provable thresholds for topological codes
• Scalable Neural Network Decoders for Higher Dimensional Quantum Codes

Cited by in corpus (51)

• Optimizing Quantum Error Correction Codes with Reinforcement Learning
• Machine Learning for Quantum Matter
• Model-Free Quantum Control with Reinforcement Learning
• When does reinforcement learning stand out in quantum control? A comparative study on state preparation
• Learning to Decode the Surface Code with a Recurrent, Transformer-Based Neural Network
• Reinforcement Learning Decoders for Fault-Tolerant Quantum Computation
• Geometry of learning neural quantum states
• Hamiltonian Learning for Quantum Error Correction
• Real-Time Decoding for Fault-Tolerant Quantum Computing: Progress, Challenges and Outlook
• A Reinforcement Learning approach for Quantum State Engineering
• Decoding algorithms for surface codes
• Scalable Neural Decoder for Topological Surface Codes
• Quantum enhancements for deep reinforcement learning in large spaces
• Improving the dynamics of quantum sensors with reinforcement learning
• Drawing Phase Diagrams of Random Quantum Systems by Deep Learning the Wave Functions
• Self-Correcting Quantum Many-Body Control using Reinforcement Learning with Tensor Networks
• A scalable and fast artificial neural network syndrome decoder for surface codes
• Artificial Intelligence for Quantum Computing
• Neural-Network Decoders for Measurement Induced Phase Transitions
• Reinforcement learning for optimal error correction of toric codes
• Unsupervised learning using topological data augmentation
• Deep Q-learning decoder for depolarizing noise on the toric code
• Expressive power of complex-valued restricted Boltzmann machines for solving non-stoquastic Hamiltonians
• Techniques for combining fast local decoders with global decoders under circuit-level noise
• Quantum variational learning for quantum error-correcting codes
• Simulating Non-Markovian Quantum Dynamics on NISQ Computers Using the Hierarchical Equations of Motion
• Simultaneous Discovery of Quantum Error Correction Codes and Encoders with a Noise-Aware Reinforcement Learning Agent
• Quantum reinforcement learning in continuous action space
• Determination of the Semion Code Threshold using Neural Decoders
• Taming quantum systems: A tutorial for using shortcuts-to-adiabaticity, quantum optimal control, and reinforcement learning
• Data-driven decoding of quantum error correcting codes using graph neural networks
• Neural network decoder for near-term surface-code experiments
• Detection of entangled states supported by reinforcement learning
• Quantum error reduction with deep neural network applied at the post-processing stage
• Creation and manipulation of quantized vortices in Bose-Einstein condensates using reinforcement learning
• Error-rate-agnostic decoding of topological stabilizer codes
• Improved Belief Propagation Decoding Algorithms for Surface Codes
• An RNN-policy gradient approach for quantum architecture search
• Optimizing thermodynamic trajectories using evolutionary and gradient-based reinforcement learning
• Unsupervised interpretable learning of topological indices invariant under permutations of atomic bands
• Improving robustness of quantum feedback control with reinforcement learning
• Quantum Circuit Discovery for Fault-Tolerant Logical State Preparation with Reinforcement Learning
• Quantum error mitigation in the regime of high noise using deep neural network: Trotterized dynamics
• A NEAT Quantum Error Decoder
• Enhancing Quantum Memory Lifetime with Measurement-Free Local Error Correction and Reinforcement Learning
• Quantum Compiling by Deep Reinforcement Learning
• GAN decoder on a quantum toric code for noise-robust quantum teleportation
• On the interpretability of neural network decoders
• Learning Feedback Mechanisms for Measurement-Based Variational Quantum State Preparation
• Hybrid Reward-Driven Reinforcement Learning for Efficient Quantum Circuit Synthesis
• Neural Decoders for Universal Quantum Algorithms