Supervised Learning for Dynamical System Learning

Abstract

Recently there has been substantial interest in spectral methods for learning dynamical systems. These methods are popular since they often offer a good tradeoff between computational and statistical efficiency. Unfortunately, they can be difficult to use and extend in practice: e.g., they can make it difficult to incorporate prior information such as sparsity or structure. To address this problem, we present a new view of dynamical system learning: we show how to learn dynamical systems by solving a sequence of ordinary supervised learning problems, thereby allowing users to incorporate prior knowledge via standard techniques such as L1 regularization. Many existing spectral methods are special cases of this new framework, using linear regression as the supervised learner. We demonstrate the effectiveness of our framework by showing examples where nonlinear regression or lasso let us learn better state representations than plain linear regression does; the correctness of these instances follows directly from our general analysis.

Cited by in corpus (12)

• Predictive State Recurrent Neural Networks
• Learning to Filter with Predictive State Inference Machines
• Learning and Inference in Hilbert Space with Quantum Graphical Models
• Predictive-State Decoders: Encoding the Future into Recurrent Networks
• Learning Deep Hidden Nonlinear Dynamics from Aggregate Data
• Recurrent Predictive State Policy Networks
• Learning Stochastic Behaviour from Aggregate Data
• Kernel Instrumental Variable Regression
• Practical Learning of Predictive State Representations
• Quantum Tensor Networks, Stochastic Processes, and Weighted Automata
• Invertible generalized synchronization: A putative mechanism for implicit learning in biological and artificial neural systems
• Learning General Latent-Variable Graphical Models with Predictive Belief Propagation