Linear Estimating Equations for Exponential Families with Application to Gaussian Linear Concentration Models

Abstract

In many families of distributions, maximum likelihood estimation is intractable because the normalization constant for the density which enters into the likelihood function is not easily available. The score matching estimator of Hyvärinen (2005) provides an alternative where this normalization constant is not required. The corresponding estimating equations become linear for an exponential family. The score matching estimator is shown to be consistent and asymptotically normally distributed for such models, although not necessarily efficient. Gaussian linear concentration models are examples of such families. For linear concentration models that are also linear in the covariance we show that the score matching estimator is identical to the maximum likelihood estimator, hence in such cases it is also efficient. Gaussian graphical models and graphical models with symmetries form particularly interesting subclasses of linear concentration models and we investigate the potential use of the score matching estimator for this case.

References in corpus (2)

• Regularized estimation of large covariance matrices
• Sparse permutation invariant covariance estimation

Cited by in corpus (6)

• The Maximum Likelihood Threshold of a Graph
• Generalized Score Matching for General Domains
• Information criteria for non-normalized models
• Existence and Uniqueness of Proper Scoring Rules
• Comparisons of Hyvärinen and pairwise estimators in two simple linear time series models
• Denoising Score Matching with Random Fourier Features