Formulation and Learning Of The Forecasting Problem

Abstract

Machine learning has opened new frontiers in purely data-driven forecasting of dynamical systems; the resulting methods are showing considerable promise in some application domains. However, in contrast to model-driven algorithms, analysis of these data-driven methods is poorly developed. In this paper we address this issue, developing a theory to underpin data-driven methods to solve forecasting problems. The theoretical framework comprises two key components: (i) establishing the existence of a mapping defining the forecasting problem; (ii) showing that it is possible to approximate this mapping by supervised learning techniques, to arbitrary accuracy. We work in the continuous-time and noise-free setting as this defines a simple canonical setting in which to establish an underpinning theory; the continuous-time setting leads to an operator formulation of the problem in which the underlying forecasting map is defined on pathspace. As a byproduct of our framework we show that analogous existence and approximation results may be obtained for the smoothing problem of recovering unobserved components of dynamical systems, from observed components, using supervised learning. The theoretical results, for both forecasting and smoothing, are illustrated with experiments studying the Lorenz `63, Lorenz `96 and Kuramoto-Sivashinsky dynamical systems.