A Computationally Efficient Method for Learning Exponential Family Distributions

Abstract

We consider the question of learning the natural parameters of a $k$ parameter minimal exponential family from i.i.d. samples in a computationally and statistically efficient manner. We focus on the setting where the support as well as the natural parameters are appropriately bounded. While the traditional maximum likelihood estimator for this class of exponential family is consistent, asymptotically normal, and asymptotically efficient, evaluating it is computationally hard. In this work, we propose a computationally efficient estimator that is consistent as well as asymptotically normal under mild conditions. We provide finite sample guarantees to achieve an (${\ell }_2$) error of $\alpha$ in the parameter estimation with sample complexity $O(\mathrm{poly}(k/\alpha ))$ and computational complexity $O(\mathrm{poly}(k/\alpha ))$. To establish these results, we show that, at the population level, our method can be viewed as the maximum likelihood estimation of a re-parameterized distribution belonging to the same class of exponential family.

References in corpus (10)

• Latent variable graphical model selection via convex optimization
• Learning Exponential Families in High-Dimensions: Strong Convexity and Sparsity
• Learning Latent Variable Gaussian Graphical Models
• Estimating Differential Latent Variable Graphical Models with Applications to Brain Connectivity
• Minimum Stein Discrepancy Estimators
• Exponential Family Estimation via Adversarial Dynamics Embedding
• Learning Some Popular Gaussian Graphical Models without Condition Number Bounds
• Telescoping Density-Ratio Estimation
• Outlier-Robust Learning of Ising Models Under Dobrushin's Condition
• Learning Continuous Exponential Families Beyond Gaussian

Cited by in corpus (1)

• The duo Bregman and Fenchel-Young divergences