Provable Tensor Methods for Learning Mixtures of Generalized Linear Models

Abstract

We consider the problem of learning mixtures of generalized linear models (GLM) which arise in classification and regression problems. Typical learning approaches such as expectation maximization (EM) or variational Bayes can get stuck in spurious local optima. In contrast, we present a tensor decomposition method which is guaranteed to correctly recover the parameters. The key insight is to employ certain feature transformations of the input, which depend on the input generative model. Specifically, we employ score function tensors of the input and compute their cross-correlation with the response variable. We establish that the decomposition of this tensor consistently recovers the parameters, under mild non-degeneracy conditions. We demonstrate that the computational and sample complexity of our method is a low order polynomial of the input and the latent dimensions.

To appear in Proceeding of AI and Statistics (AISTATS) 2016

References in corpus (2)

• Bayesian Hierarchical Mixtures of Experts
• Sample Complexity Analysis for Learning Overcomplete Latent Variable Models through Tensor Methods

Cited by in corpus (6)

• Complete Dictionary Recovery over the Sphere I: Overview and the Geometric Picture
• Complete Dictionary Recovery over the Sphere II: Recovery by Riemannian Trust-region Method
• When Are Nonconvex Problems Not Scary?
• Convolutional Phase Retrieval via Gradient Descent
• Mixture Conditional Regression with Ultrahigh Dimensional Text Data for Estimating Extralegal Factor Effects
• Score Function Features for Discriminative Learning