Learning Mixtures of Smooth Product Distributions: Identifiability and Algorithm

Abstract

We study the problem of learning a mixture model of non-parametric product distributions. The problem of learning a mixture model is that of finding the component distributions along with the mixing weights using observed samples generated from the mixture. The problem is well-studied in the parametric setting, i.e., when the component distributions are members of a parametric family -- such as Gaussian distributions. In this work, we focus on multivariate mixtures of non-parametric product distributions and propose a two-stage approach which recovers the component distributions of the mixture under a smoothness condition. Our approach builds upon the identifiability properties of the canonical polyadic (low-rank) decomposition of tensors, in tandem with Fourier and Shannon-Nyquist sampling staples from signal processing. We demonstrate the effectiveness of the approach on synthetic and real datasets.

accepted to appear in AISTATS 2019

Cited by in corpus (4)

• Stochastic Mirror Descent for Low-Rank Tensor Decomposition Under Non-Euclidean Losses
• Recovering Joint Probability of Discrete Random Variables from Pairwise Marginals
• Low-rank Characteristic Tensor Density Estimation Part I: Foundations
• Block-Randomized Stochastic Proximal Gradient for Low-Rank Tensor Factorization