Hidden Markov Nonlinear ICA: Unsupervised Learning from Nonstationary Time Series
arXiv:2006.12107
Abstract
Recent advances in nonlinear Independent Component Analysis (ICA) provide a principled framework for unsupervised feature learning and disentanglement. The central idea in such works is that the latent components are assumed to be independent conditional on some observed auxiliary variables, such as the time-segment index. This requires manual segmentation of data into non-stationary segments which is computationally expensive, inaccurate and often impossible. These models are thus not fully unsupervised. We remedy these limitations by combining nonlinear ICA with a Hidden Markov Model, resulting in a model where a latent state acts in place of the observed segment-index. We prove identifiability of the proposed model for a general mixing nonlinearity, such as a neural network. We also show how maximum likelihood estimation of the model can be done using the expectation-maximization algorithm. Thus, we achieve a new nonlinear ICA framework which is unsupervised, more efficient, as well as able to model underlying temporal dynamics.
arXiv commentAccepted for publication at UAI 2020
References in corpus (8)
- Adam: A Method for Stochastic Optimization
- Auto-Encoding Variational Bayes
- NICE: Non-linear Independent Components Estimation
- Identifiability of parameters in latent structure models with many observed variables
- Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICA
- Variational Autoencoders and Nonlinear ICA: A Unifying Framework
- Disentanglement by Nonlinear ICA with General Incompressible-flow Networks (GIN)
- Stochastic Variational Inference for Hidden Markov Models
Cited by in corpus (4)
- Disentangling Identifiable Features from Noisy Data with Structured Nonlinear ICA
- Modelling multivariate spatio-temporal data with identifiable variational autoencoders
- Nonlinear blind source separation exploiting spatial nonstationarity
- Path classification by stochastic linear recurrent neural networks