A Tensor Network Approach to Finite Markov Decision Processes
arXiv:2002.05185
Abstract
Tensor network (TN) techniques - often used in the context of quantum many-body physics - have shown promise as a tool for tackling machine learning (ML) problems. The application of TNs to ML, however, has mostly focused on supervised and unsupervised learning. Yet, with their direct connection to hidden Markov chains, TNs are also naturally suited to Markov decision processes (MDPs) which provide the foundation for reinforcement learning (RL). Here we introduce a general TN formulation of finite, episodic and discrete MDPs. We show how this formulation allows us to exploit algorithms developed for TNs for policy optimisation, the key aim of RL. As an application we consider the issue - formulated as an RL problem - of finding a stochastic evolution that satisfies specific dynamical conditions, using the simple example of random walk excursions as an illustration.
10 pages, 2 figures
References in corpus (23)
- Trust Region Policy Optimization
- Area laws for the entanglement entropy - a review
- Soft Actor-Critic Algorithms and Applications
- Matrix product operator representations
- Nonequilibrium Markov processes conditioned on large deviations
- Neural-Network Approach to Dissipative Quantum Many-Body Dynamics
- Unsupervised Generative Modeling Using Matrix Product States
- Learning Thermodynamics with Boltzmann Machines
- Benchmarking Model-Based Reinforcement Learning
- Variational Quantum Monte Carlo Method with a Neural-Network Ansatz for Open Quantum Systems
- Identifying Quantum Phase Transitions using Artificial Neural Networks on Experimental Data
- Variational neural network ansatz for steady states in open quantum systems
- Quantum Entanglement in Deep Learning Architectures
- Constructing neural stationary states for open quantum many-body systems
- Exponential Decay of Correlations Implies Area Law
- Aspects of non-equilibrium in classical and quantum systems: slow relaxation and glasses, dynamical large deviations, quantum non-ergodicity, and open quantum dynamics
- Finite automata for caching in matrix product algorithms
- Ergodicity and large deviations in physical systems with stochastic dynamics
- TensorNetwork for Machine Learning
- Entanglement and tensor network states
- Generative Tensor Network Classification Model for Supervised Machine Learning
- Classical stochastic dynamics and continuous matrix product states: gauge transformations, conditioned and driven processes, and equivalence of trajectory ensembles
- Approximating local properties by tensor network states with constant bond dimension