Nonparametric Estimation of Multi-View Latent Variable Models

Abstract

Spectral methods have greatly advanced the estimation of latent variable models, generating a sequence of novel and efficient algorithms with strong theoretical guarantees. However, current spectral algorithms are largely restricted to mixtures of discrete or Gaussian distributions. In this paper, we propose a kernel method for learning multi-view latent variable models, allowing each mixture component to be nonparametric. The key idea of the method is to embed the joint distribution of a multi-view latent variable into a reproducing kernel Hilbert space, and then the latent parameters are recovered using a robust tensor power method. We establish that the sample complexity for the proposed method is quadratic in the number of latent components and is a low order polynomial in the other relevant parameters. Thus, our non-parametric tensor approach to learning latent variable models enjoys good sample and computational efficiencies. Moreover, the non-parametric tensor power method compares favorably to EM algorithm and other existing spectral algorithms in our experiments.

References in corpus (2)

• A Spectral Algorithm for Latent Dirichlet Allocation
• Spectral dimensionality reduction for HMMs

Cited by in corpus (7)

• Beating the Perils of Non-Convexity: Guaranteed Training of Neural Networks using Tensor Methods
• Long-term Forecasting using Higher Order Tensor RNNs
• Score Function Features for Discriminative Learning: Matrix and Tensor Framework
• Sample Complexity Analysis for Learning Overcomplete Latent Variable Models through Tensor Methods
• Provable Tensor Methods for Learning Mixtures of Generalized Linear Models
• Training Input-Output Recurrent Neural Networks through Spectral Methods
• Reinforcement Learning in Rich-Observation MDPs using Spectral Methods