Convergence Rates for Gaussian Mixtures of Experts

Abstract

We provide a theoretical treatment of over-specified Gaussian mixtures of experts with covariate-free gating networks. We establish the convergence rates of the maximum likelihood estimation (MLE) for these models. Our proof technique is based on a novel notion of \emph{algebraic independence} of the expert functions. Drawing on optimal transport theory, we establish a connection between the algebraic independence and a certain class of partial differential equations (PDEs). Exploiting this connection allows us to derive convergence rates and minimax lower bounds for parameter estimation.

81 pages

References in corpus (2)

• Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
• Consistency of the MLE under mixture models

Cited by in corpus (5)

• Minimax Confidence Intervals for the Sliced Wasserstein Distance
• On the Minimax Optimality of the EM Algorithm for Learning Two-Component Mixed Linear Regression
• Uniform Consistency in Nonparametric Mixture Models
• A non-asymptotic approach for model selection via penalization in high-dimensional mixture of experts models
• Learning in Gated Neural Networks