Quantum machine learning with glow for episodic tasks and decision games

Abstract

We consider a general class of models, where a reinforcement learning (RL) agent learns from cyclic interactions with an external environment via classical signals. Perceptual inputs are encoded as quantum states, which are subsequently transformed by a quantum channel representing the agent's memory, while the outcomes of measurements performed at the channel's output determine the agent's actions. The learning takes place via stepwise modifications of the channel properties. They are described by an update rule that is inspired by the projective simulation (PS) model and equipped with a glow mechanism that allows for a backpropagation of policy changes, analogous to the eligibility traces in RL and edge glow in PS. In this way, the model combines features of PS with the ability for generalization, offered by its physical embodiment as a quantum system. We apply the agent to various setups of an invasion game and a grid world, which serve as elementary model tasks allowing a direct comparison with a basic classical PS agent.

20 pages, 14 figures

References in corpus (13)

• Quantum Machine Learning
• An introduction to quantum machine learning
• Training Schrödinger's cat: quantum optimal control
• The quest for a Quantum Neural Network
• Quantum reinforcement learning
• Fidelity of quantum operations
• Quantum brachistochrone curves as geodesics: obtaining accurate control protocols for time-optimal quantum gates
• Local observation of antibunching in a trapped Fermi gas
• Quantum walks on graphs representing the firing patterns of a quantum neural network
• Projective simulation with generalization
• Hybrid Optimization Schemes for Quantum Control
• Quantum-enhanced deliberation of learning agents using trapped ions
• Time-optimal bath-induced unitaries by Zermelo navigation: speed limit and non-Markovian quantum computation

Cited by in corpus (7)

• Reinforcement learning for autonomous preparation of Floquet-engineered states: Inverting the quantum Kapitza oscillator
• Classification with Quantum Machine Learning: A Survey
• Machine learning \& artificial intelligence in the quantum domain
• On the convergence of projective-simulation-based reinforcement learning in Markov decision processes
• Noisy three-player dilemma game: Robustness of the quantum advantage
• Experimental Demonstration on Quantum Sensitivity to Available Information in Decision Making
• A Projective Simulation Scheme for Partially-Observable Multi-Agent Systems