Quantum Logic Gate Synthesis as a Markov Decision Process

Abstract

Reinforcement learning has witnessed recent applications to a variety of tasks in quantum programming. The underlying assumption is that those tasks could be modeled as Markov Decision Processes (MDPs). Here, we investigate the feasibility of this assumption by exploring its consequences for two fundamental tasks in quantum programming: state preparation and gate compilation. By forming discrete MDPs, focusing exclusively on the single-qubit case (both with and without noise), we solve for the optimal policy exactly through policy iteration. We find optimal paths that correspond to the shortest possible sequence of gates to prepare a state, or compile a gate, up to some target accuracy. As an example, we find sequences of $H$ and $T$ gates with length as small as $11$ producing $\sim 99\%$ fidelity for states of the form $(HT{)}^n|0⟩$ with values as large as $n=10^{10}$. In the presence of gate noise, we demonstrate how the optimal policy adapts to the effects of noisy gates in order to achieve a higher state fidelity. Our work shows that one can meaningfully impose a discrete, stochastic and Markovian nature to a continuous, deterministic and non-Markovian quantum evolution, and provides theoretical insight into why reinforcement learning may be successfully used to find optimally short gate sequences in quantum programming.

References in corpus (29)

• Error mitigation for short-depth quantum circuits
• An Introduction to Deep Reinforcement Learning
• Synthesis of Quantum Logic Circuits
• Efficient variational quantum simulator incorporating active error minimisation
• Optimal Quantum Circuits for General Two-Qubit Gates
• Dynamically Error-Corrected Gates for Universal Quantum Computation
• Quantum agents in the Gym: a variational quantum algorithm for deep Q-learning
• Random decoupling schemes for quantum dynamical control and error suppression
• Model-Free Quantum Control with Reinforcement Learning
• When does reinforcement learning stand out in quantum control? A comparative study on state preparation
• Deep Reinforcement Learning for Quantum Gate Control
• Machine learning of noise-resilient quantum circuits
• Gradient-based optimal control of open quantum systems using quantum trajectories and automatic differentiation
• Coherent Transport of Quantum States by Deep Reinforcement Learning
• Unified approach to data-driven quantum error mitigation
• Reinforcement Learning assisted Quantum Optimization
• Reinforcement learning for autonomous preparation of Floquet-engineered states: Inverting the quantum Kapitza oscillator
• Extending quantum probabilistic error cancellation by noise scaling
• A Tutorial on Optimal Control and Reinforcement Learning methods for Quantum Technologies
• Measurement-based adaptation protocol with quantum reinforcement learning
• A Reinforcement Learning approach for Quantum State Engineering
• Quantum machine learning and quantum biomimetics: A perspective
• Setting up experimental Bell test with reinforcement learning
• Machine Learning and Quantum Devices
• Fixed-Depth Two-Qubit Circuits and the Monodromy Polytope
• Quantum exploration algorithms for multi-armed bandits
• Quantum Machine Learning Implementations: Proposals and Experiments
• Automated quantum programming via reinforcement learning for combinatorial optimization
• Multi-armed quantum bandits: Exploration versus exploitation when learning properties of quantum states

Cited by in corpus (4)

• Topological Quantum Compiling with Reinforcement Learning
• Reinforcement learning for optimization of variational quantum circuit architectures
• Assessing and Advancing the Potential of Quantum Computing: A NASA Case Study
• High-Precision Multi-Qubit Clifford+T Synthesis by Unitary Diagonalization