Quantum-accessible reinforcement learning beyond strictly epochal environments

Abstract

In recent years, quantum-enhanced machine learning has emerged as a particularly fruitful application of quantum algorithms, covering aspects of supervised, unsupervised and reinforcement learning. Reinforcement learning offers numerous options of how quantum theory can be applied, and is arguably the least explored, from a quantum perspective. Here, an agent explores an environment and tries to find a behavior optimizing some figure of merit. Some of the first approaches investigated settings where this exploration can be sped-up, by considering quantum analogs of classical environments, which can then be queried in superposition. If the environments have a strict periodic structure in time (i.e. are strictly episodic), such environments can be effectively converted to conventional oracles encountered in quantum information. However, in general environments, we obtain scenarios that generalize standard oracle tasks. In this work we consider one such generalization, where the environment is not strictly episodic, which is mapped to an oracle identification setting with a changing oracle. We analyze this case and show that standard amplitude-amplification techniques can, with minor modifications, still be applied to achieve quadratic speed-ups, and that this approach is optimal for certain settings. This results constitutes one of the first generalizations of quantum-accessible reinforcement learning.

arXiv comment8+9 pages, 2 figures

References in corpus (19)

• The Quantum Internet
• Quantum Machine Learning
• Quantum Mechanics helps in searching for a needle in a haystack
• Quantum algorithm for solving linear systems of equations
• Supervised learning with quantum enhanced feature spaces
• Tight bounds on quantum searching
• Grover's quantum searching algorithm is optimal
• Quantum computers can search rapidly by using almost any transformation
• Quantum Internet: Networking Challenges in Distributed Quantum Computing
• Quantum-enhanced machine learning
• Active learning machine learns to create new quantum experiments
• Fixed-point quantum search with an optimal number of queries
• Quantum speedup for active learning agents
• Experimental quantum speed-up in reinforcement learning agents
• Sampling-based sublinear low-rank matrix arithmetic framework for dequantizing quantum machine learning
• Advances in Quantum Reinforcement Learning
• Speeding-up the decision making of a learning agent using an ion trap quantum processor
• Benchmarking projective simulation in navigation problems
• Quantum Query Algorithms are Completely Bounded Forms

Cited by in corpus (2)

• Quantum agents in the Gym: a variational quantum algorithm for deep Q-learning
• Performance analysis of a hybrid agent for quantum-accessible reinforcement learning