The Quantum Cartpole: A benchmark environment for non-linear reinforcement learning

Abstract

Feedback-based control is the de-facto standard when it comes to controlling classical stochastic systems and processes. However, standard feedback-based control methods are challenged by quantum systems due to measurement induced backaction and partial observability. Here we remedy this by using weak quantum measurements and model-free reinforcement learning agents to perform quantum control. By comparing control algorithms with and without state estimators to stabilize a quantum particle in an unstable state near a local potential energy maximum, we show how a trade-off between state estimation and controllability arises. For the scenario where the classical analogue is highly nonlinear, the reinforcement learned controller has an advantage over the standard controller. Additionally, we demonstrate the feasibility of using transfer learning to develop a quantum control agent trained via reinforcement learning on a classical surrogate of the quantum control problem. Finally, we present results showing how the reinforcement learning control strategy differs from the classical controller in the non-linear scenarios.

arXiv comment22 pages, 11 figures, 2 tables

References in corpus (17)

• Quantum Simulation
• OpenAI Gym
• Feedback-control of quantum systems using continuous state-estimation
• Optimal control technique for Many Body Quantum Systems dynamics
• Reinforcement Learning in Different Phases of Quantum Control
• Reinforcement Learning with Neural Networks for Quantum Feedback
• Optimizing Quantum Error Correction Codes with Reinforcement Learning
• NetKet: A Machine Learning Toolkit for Many-Body Quantum Systems
• Evolutionary Algorithms for Hard Quantum Control
• Global optimization of quantum dynamics with AlphaZero deep exploration
• Measurement Based Feedback Quantum Control With Deep Reinforcement Learning for Double-well Non-linear Potential
• Deep Reinforcement Learning for Quantum State Preparation with Weak Nonlinear Measurements
• Reinforcement Learning Decoders for Fault-Tolerant Quantum Computation
• Self-Organizing Maps for Storage and Transfer of Knowledge in Reinforcement Learning
• Deep Reinforcement Learning Control of Quantum Cartpoles
• Ctrl-Z: Recovering from Instability in Reinforcement Learning
• Prospects of reinforcement learning for the simultaneous damping of many mechanical modes