Unsupervised model-free representation learning

Abstract

Numerous control and learning problems face the situation where sequences of high-dimensional highly dependent data are available but no or little feedback is provided to the learner, which makes any inference rather challenging. To address this challenge, we formulate the following problem. Given a series of observations $X_0,\dots ,X_n$ coming from a large (high-dimensional) space $\mathcal{X}$, find a representation function $f$ mapping $\mathcal{X}$ to a finite space $\mathcal{Y}$ such that the series $f(X_0),\dots ,f(X_n)$ preserves as much information as possible about the original time-series dependence in $X_0,\dots ,X_n$. We show that, for stationary time series, the function $f$ can be selected as the one maximizing a certain information criterion that we call time-series information. Some properties of this functions are investigated, including its uniqueness and consistency of its empirical estimates. Implications for the problem of optimal control are presented.

The update is the journal version appearing in IEEE IT transactions under the title "Time-series information and unsupervised learning of representations." This version includes important corrections and new results. Some of the results (presented in previous versions) were reported at ISIT'13 and ALT'13

References in corpus (6)

• The information bottleneck method
• Computational Mechanics: Pattern and Prediction, Structure and Simplicity
• Selecting the State-Representation in Reinforcement Learning
• Pattern Recognition for Conditionally Independent Data
• Optimal Regret Bounds for Selecting the State Representation in Reinforcement Learning
• Hypotheses testing on infinite random graphs