Enforcing constraints for time series prediction in supervised, unsupervised and reinforcement learning
arXiv:1905.07501
Abstract
We assume that we are given a time series of data from a dynamical system and our task is to learn the flow map of the dynamical system. We present a collection of results on how to enforce constraints coming from the dynamical system in order to accelerate the training of deep neural networks to represent the flow map of the system as well as increase their predictive ability. In particular, we provide ways to enforce constraints during training for all three major modes of learning, namely supervised, unsupervised and reinforcement learning. In general, the dynamic constraints need to include terms which are analogous to memory terms in model reduction formalisms. Such memory terms act as a restoring force which corrects the errors committed by the learned flow map during prediction. For supervised learning, the constraints are added to the objective function. For the case of unsupervised learning, in particular generative adversarial networks, the constraints are introduced by augmenting the input of the discriminator. Finally, for the case of reinforcement learning and in particular actor-critic methods, the constraints are added to the reward function. In addition, for the reinforcement learning case, we present a novel approach based on homotopy of the action-value function in order to stabilize and accelerate training. We use numerical results for the Lorenz system to illustrate the various constructions.
arXiv comment30 pages, 5 figures
References in corpus (10)
- Continuous control with deep reinforcement learning
- DGM: A deep learning algorithm for solving partial differential equations
- Solving high-dimensional partial differential equations using deep learning
- Neural Ordinary Differential Equations
- Towards Principled Methods for Training Generative Adversarial Networks
- Data-assisted reduced-order modeling of extreme events in complex dynamical systems
- Nonparametric forecasting of low-dimensional dynamical systems
- Connecting Generative Adversarial Networks and Actor-Critic Methods
- Enforcing constraints for interpolation and extrapolation in Generative Adversarial Networks
- Deep Learning for Reward Design to Improve Monte Carlo Tree Search in ATARI Games