A Tensor Network Approach to Finite Markov Decision Processes

Abstract

Tensor network (TN) techniques - often used in the context of quantum many-body physics - have shown promise as a tool for tackling machine learning (ML) problems. The application of TNs to ML, however, has mostly focused on supervised and unsupervised learning. Yet, with their direct connection to hidden Markov chains, TNs are also naturally suited to Markov decision processes (MDPs) which provide the foundation for reinforcement learning (RL). Here we introduce a general TN formulation of finite, episodic and discrete MDPs. We show how this formulation allows us to exploit algorithms developed for TNs for policy optimisation, the key aim of RL. As an application we consider the issue - formulated as an RL problem - of finding a stochastic evolution that satisfies specific dynamical conditions, using the simple example of random walk excursions as an illustration.

10 pages, 2 figures

References in corpus (7)

• Neural-Network Approach to Dissipative Quantum Many-Body Dynamics
• Benchmarking Model-Based Reinforcement Learning
• Variational Quantum Monte Carlo Method with a Neural-Network Ansatz for Open Quantum Systems
• Variational neural network ansatz for steady states in open quantum systems
• Constructing neural stationary states for open quantum many-body systems
• Finite automata for caching in matrix product algorithms
• TensorNetwork for Machine Learning