Reinforcement learning for autonomous preparation of Floquet-engineered states: Inverting the quantum Kapitza oscillator

Abstract

I demonstrate the potential of reinforcement learning (RL) to prepare quantum states of strongly periodically driven non-linear single-particle models. The ability of Q-Learning to control systems far away from equilibrium is exhibited by steering the quantum Kapitza oscillator to the Floquet-engineered stable inverted position in the presence of a strong periodic drive within several shaking cycles. The study reveals the potential of the intra-period (micromotion) dynamics, often neglected in Floquet engineering, to take advantage over pure stroboscopic control at moderate drive frequencies. Without any knowledge about the underlying physical system, the algorithm is capable of learning solely from tried protocols and directly from simulated noisy quantum measurement data, and is stable to noise in the initial state, and sources of random failure events in the control sequence. Model-free RL can provide new insights into automating experimental setups for out-of-equilibrium systems undergoing complex dynamics, with potential applications in quantum information, quantum optics, ultracold atoms, trapped ions, and condensed matter.

arXiv commentLegends for the movies referred to in the paper are available on https://mgbukov.github.io/RL_kapitza/

References in corpus (87)

• A variational eigenvalue solver on a quantum processor
• Photonic Floquet Topological Insulators
• From Quantum Chaos and Eigenstate Thermalization to Statistical Mechanics and Thermodynamics
• The theory of variational hybrid quantum-classical algorithms
• Solving the Quantum Many-Body Problem with Artificial Neural Networks
• Experimental realisation of the topological Haldane model
• Photovoltaic Hall effect in graphene
• Realization of the Hofstadter Hamiltonian with ultracold atoms in optical lattices
• Universal High-Frequency Behavior of Periodically Driven Systems: from Dynamical Stabilization to Floquet Engineering
• Atomic quantum gases in periodically driven optical lattices
• Light-induced gauge fields for ultracold atoms
• Deep Reinforcement Learning for De-Novo Drug Design
• Realizing the Harper Hamiltonian with Laser-Assisted Tunneling in Optical Lattices
• Transport properties of non-equilibrium systems under the application of light: Photo-induced quantum Hall insulators without Landau levels
• Measuring the Chern number of Hofstadter bands with ultracold bosonic atoms
• The cold atom Hubbard toolbox
• Spin-orbit coupling in quantum gases
• Periodically-driven quantum systems: Effective Hamiltonians and engineered gauge fields
• Training Schrödinger's cat: quantum optimal control
• Observation of chiral edge states with neutral fermions in synthetic Hall ribbons
• Shaped-pulse optimisation of coherent soft-x-rays
• Quantum simulation of frustrated magnetism in triangular optical lattices
• Visualizing edge states with an atomic Bose gas in the quantum Hall regime
• Tunable gauge potential for neutral and spinless particles in driven lattices
• Synthetic gauge fields in synthetic dimensions
• Superfluid-insulator transition in a periodically driven optical lattice
• Feedback-control of quantum systems using continuous state-estimation
• Observation of the Meissner effect with ultracold atoms in bosonic ladders
• Chiral groundstate currents of interacting photons in a synthetic magnetic field
• Experimental reconstruction of the Berry curvature in a topological Bloch band
• Floquet-Magnus Theory and Generic Transient Dynamics in Periodically Driven Many-Body Quantum Systems
• QuSpin: a Python Package for Dynamics and Exact Diagonalisation of Quantum Many Body Systems part I: spin chains
• Chopped random-basis quantum optimization
• Cloud Quantum Computing of an Atomic Nucleus
• Rigorous Bound on Energy Absorption and Generic Relaxation in Periodically Driven Quantum Systems
• Non-Abelian gauge fields and topological insulators in shaken optical lattices
• Recent progress in many-body localization
• Engineering Ising-XY spin models in a triangular lattice via tunable artificial gauge fields
• Reinforcement Learning in Different Phases of Quantum Control
• Active learning machine learns to create new quantum experiments
• Topologically Robust Transport of Photons in a Synthetic Gauge Field
• Artificial gauge fields in materials and engineered systems
• Observation of photon-assisted tunneling in optical lattices
• Ultrafast and reversible control of the exchange interaction in Mott insulators
• Flow Navigation by Smart Microswimmers via Reinforcement Learning
• Reinforcement Learning with Neural Networks for Quantum Feedback
• Observation of Bose-Einstein Condensation in a Strong Synthetic Magnetic Field
• Quantum speedup for active learning agents
• Magnetically generated spin-orbit coupling for ultracold atoms
• Tunable Spin-Orbit Coupling via Strong Driving in Ultracold Atom Systems
• Coherent control using adaptive learning algorithms
• Prethermal Floquet Steady States and Instabilities in the Periodically Driven, Weakly Interacting Bose-Hubbard Model
• Fidelity-based Probabilistic Q-learning for Control of Quantum Systems
• Frustrated quantum antiferromagnetism with ultracold bosons in a triangular lattice
• Experimental feedback control of quantum systems using weak measurements
• Floquet prethermalization and regimes of heating in a periodically driven, interacting quantum system
• Enhancement and sign change of magnetic correlations in a driven quantum many-body system
• Adiabatic Perturbation Theory and Geometry of Periodically-Driven Systems
• Scattering Theory for Floquet-Bloch States
• Heating and many-body resonances in a periodically driven two-band system
• Interaction dependent heating and atom loss in a periodically driven optical lattice
• Creating State-Dependent Lattices for Ultracold Fermions by Magnetic Gradient Modulation
• Measuring topological invariants in photonic systems
• Dynamical Stability of a Many-body Kapitza Pendulum
• Time Dependent Floquet Theory and Absence of an Adiabatic Limit
• Floquet dynamics in driven Fermi-Hubbard systems
• Quantum Optimal Control in a Chopped Basis: Applications in Control of Bose-Einstein Condensates
• Automatic spin-chain learning to explore the quantum speed limit
• Phase transition in protocols minimizing work fluctuations
• Floquet analysis of a quantum system with modulated periodic driving
• Parametric Instability Rates in Periodically-Driven Band Systems
• Measurement-based adaptation protocol with quantum reinforcement learning
• Controlling the Floquet state population and observing micromotion in a periodically driven two-body quantum system
• Population dynamics in Floquet realisation of Harper-Hofstadter Hamiltonian
• Avoided level crossing spectroscopy with dressed matter waves
• Resonant thermalization of periodically driven strongly correlated electrons
• Spin polarization through Floquet resonances in a driven central spin model
• Quantum Optimal Control: Landscape Structure and Topology
• Stroboscopic versus non-stroboscopic dynamics in the Floquet realization of the Harper-Hofstadter Hamiltonian
• A Survey of Quantum Lyapunov Control Methods
• Observation of topological Bloch-state defects and their merging transition
• Autonomous Calibration of Single Spin Qubit Operations
• Broken symmetry in a two-qubit quantum control landscape
• Tunneling control and localization for Bose-Einstein condensates in a frequency modulated optical lattice
• Quantum machine learning with glow for episodic tasks and decision games
• Control of the Oscillatory Interlayer Exchange Interaction with Terahertz Radiation
• Taking gradients through experiments: LSTMs and memory proximal policy optimization for black-box quantum control

Cited by in corpus (54)

• Machine learning and the physical sciences
• A high-bias, low-variance introduction to Machine Learning for physicists
• QuSpin: a Python Package for Dynamics and Exact Diagonalisation of Quantum Many Body Systems. Part II: bosons, fermions and higher spins
• Model-Free Quantum Control with Reinforcement Learning
• When does reinforcement learning stand out in quantum control? A comparative study on state preparation
• Geometric Speed Limit of Accessible Many-Body State Preparation
• Global optimization of quantum dynamics with AlphaZero deep exploration
• Reinforcement Learning for Many-Body Ground-State Preparation Inspired by Counterdiabatic Driving
• Deep Reinforcement Learning Control of Quantum Cartpoles
• Machine Learning meets Quantum Foundations: A Brief Survey
• A Reinforcement Learning approach for Quantum State Engineering
• Quantum enhancements for deep reinforcement learning in large spaces
• Quantum machine learning and quantum biomimetics: A perspective
• A differentiable programming method for quantum control
• A reinforcement learning approach to rare trajectory sampling
• Reinforcement Learning for Digital Quantum Simulation
• Self-Correcting Quantum Many-Body Control using Reinforcement Learning with Tensor Networks
• Faster State Preparation across Quantum Phase Transition Assisted by Reinforcement Learning
• Reinforcement learning of rare diffusive dynamics
• Policy Gradient based Quantum Approximate Optimization Algorithm
• Photonic architecture for reinforcement learning
• Extreme Spin Squeezing from Deep Reinforcement Learning
• Deep reinforcement learning for universal quantum state preparation via dynamic pulse control
• A quantum system control method based on enhanced reinforcement learning
• Reinforcement learning for semi-autonomous approximate quantum eigensolver
• Universal and optimal coin sequences for high entanglement generation in 1D discrete time quantum walks
• Demonstration of a programmable optical lattice atom interferometer
• Control of Stochastic Quantum Dynamics by Differentiable Programming
• Machine classification for probe based quantum thermometry
• Benefits of Open Quantum Systems for Quantum Machine Learning
• Variational Preparation of the Sachdev-Ye-Kitaev Thermofield Double
• Quantum Logic Gate Synthesis as a Markov Decision Process
• Active Learning Approach to Optimization of Experimental Control
• Robust optimization for quantum reinforcement learning control using partial observations
• Deep reinforcement learning for preparation of thermal and prethermal quantum states
• Noise-Robust End-to-End Quantum Control using Deep Autoregressive Policy Networks
• Vector Atom Accelerometry in an Optical Lattice
• Random coordinate descent: a simple alternative for optimizing parameterized quantum circuits
• Enhanced quantum state preparation via stochastic prediction of neural network
• Creation and manipulation of quantized vortices in Bose-Einstein condensates using reinforcement learning
• Quantum reinforcement learning in the presence of thermal dissipation
• Fully-programmable universal quantum simulator with a one-dimensional quantum processor
• Time-Optimal Quantum Driving by Variational Circuit Learning
• Reinforcement Learning for Rotation Sensing with Ultracold Atoms in an Optical Lattice
• Supervised Training of Neural-Network Quantum States for the Next Nearest Neighbor Ising model
• Improving robustness of quantum feedback control with reinforcement learning
• Characterization of photoexcited states in the half-filled one-dimensional extended Hubbard model assisted by machine learning
• Fast quantum gate design with deep reinforcement learning using real-time feedback on readout signals
• Adaptive Random Quantum Eigensolver
• Minimally driven Kapitza oscillator: A pedagogical perspective from Newtonian mechanics and geometry
• Exploring fixed points and eigenstates of quantum systems with reinforcement learning
• Reinforcement Learning with Neural Networks for Quantum Multiple Hypothesis Testing
• Preparation of Many-body Ground States by Time Evolution with Variational Microscopic Magnetic Fields and Incomplete Interactions
• Reinforcement-learning based matterwave interferometer in a shaken optical lattice