Disentangling Identifiable Features from Noisy Data with Structured Nonlinear ICA

Abstract

We introduce a new general identifiable framework for principled disentanglement referred to as Structured Nonlinear Independent Component Analysis (SNICA). Our contribution is to extend the identifiability theory of deep generative models for a very broad class of structured models. While previous works have shown identifiability for specific classes of time-series models, our theorems extend this to more general temporal structures as well as to models with more complex structures such as spatial dependencies. In particular, we establish the major result that identifiability for this framework holds even in the presence of noise of unknown distribution. Finally, as an example of our framework's flexibility, we introduce the first nonlinear ICA model for time-series that combines the following very useful properties: it accounts for both nonstationarity and autocorrelation in a fully unsupervised setting; performs dimensionality reduction; models hidden states; and enables principled estimation and inference by variational maximum-likelihood.

arXiv commentAccepted for publication at NeurIPS 2021

References in corpus (9)

• Auto-Encoding Variational Bayes
• Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations
• Composing graphical models with neural networks for structured representations and fast inference
• Frequentist Consistency of Variational Bayes
• Variational Autoencoders and Nonlinear ICA: A Unifying Framework
• Learning identifiable and interpretable latent models of high-dimensional neural activity using pi-VAE
• ICE-BeeM: Identifiable Conditional Energy-Based Deep Models Based on Nonlinear ICA
• Hidden Markov Nonlinear ICA: Unsupervised Learning from Nonstationary Time Series
• Identifiability and consistent estimation of nonparametric translation hidden Markov models with general state space

Cited by in corpus (6)

• Learnable latent embeddings for joint behavioral and neural analysis
• Modelling multivariate spatio-temporal data with identifiable variational autoencoders
• Nonlinear blind source separation exploiting spatial nonstationarity
• Discovering Predictable Latent Factors for Time Series Forecasting
• Amortized backward variational inference in nonlinear state-space models
• Support and distribution inference from noisy data