Provable Tensor Methods for Learning Mixtures of Generalized Linear Models
arXiv:1412.3046
Abstract
We consider the problem of learning mixtures of generalized linear models (GLM) which arise in classification and regression problems. Typical learning approaches such as expectation maximization (EM) or variational Bayes can get stuck in spurious local optima. In contrast, we present a tensor decomposition method which is guaranteed to correctly recover the parameters. The key insight is to employ certain feature transformations of the input, which depend on the input generative model. Specifically, we employ score function tensors of the input and compute their cross-correlation with the response variable. We establish that the decomposition of this tensor consistently recovers the parameters, under mild non-degeneracy conditions. We demonstrate that the computational and sample complexity of our method is a low order polynomial of the input and the latent dimensions.
arXiv commentTo appear in Proceeding of AI and Statistics (AISTATS) 2016
References in corpus (7)
- Bayesian Hierarchical Mixtures of Experts
- What Regularized Auto-Encoders Learn from the Data Generating Distribution
- Guaranteed Non-Orthogonal Tensor Decomposition via Alternating Rank- Updates
- Least Squares Revisited: Scalable Approaches for Multi-class Prediction
- Sample Complexity Analysis for Learning Overcomplete Latent Variable Models through Tensor Methods
- Learning Mixtures of Linear Classifiers
- Nonparametric Estimation of Multi-View Latent Variable Models
Cited by in corpus (6)
- Complete Dictionary Recovery over the Sphere I: Overview and the Geometric Picture
- Complete Dictionary Recovery over the Sphere II: Recovery by Riemannian Trust-region Method
- When Are Nonconvex Problems Not Scary?
- Convolutional Phase Retrieval via Gradient Descent
- Mixture Conditional Regression with Ultrahigh Dimensional Text Data for Estimating Extralegal Factor Effects
- Score Function Features for Discriminative Learning