Generalized Score Matching for Non-Negative Data

Abstract

A common challenge in estimating parameters of probability density functions is the intractability of the normalizing constant. While in such cases maximum likelihood estimation may be implemented using numerical integration, the approach becomes computationally intensive. The score matching method of Hyvärinen [2005] avoids direct calculation of the normalizing constant and yields closed-form estimates for exponential families of continuous distributions over ${\mathbb{R}}^m$. Hyvärinen [2007] extended the approach to distributions supported on the non-negative orthant, ${\mathbb{R}}_{+}^m$. In this paper, we give a generalized form of score matching for non-negative data that improves estimation efficiency. As an example, we consider a general class of pairwise interaction models. Addressing an overlooked inexistence problem, we generalize the regularized score matching method of Lin et al. [2016] and improve its theoretical guarantees for non-negative Gaussian graphical models.

arXiv comment70 pages, 76 figures

References in corpus (4)

• Lethality and centrality in protein networks
• The huge Package for High-dimensional Undirected Graph Estimation in R
• Fast and Adaptive Sparse Precision Matrix Estimation in High Dimensions
• Square Root Graphical Models: Multivariate Generalizations of Univariate Exponential Families that Permit Positive Dependencies

Cited by in corpus (11)

• GraphPAS: Parallel Architecture Search for Graph Neural Networks
• A General Method For Automatic Discovery of Powerful Interactions In Click-Through Rate Prediction
• Characterizations of non-normalized discrete probability distributions and their application in statistics
• Minimum $L^q$-distance estimators for non-normalized parametric models
• Learning Personal Style from Few Examples
• Thresholded Adaptive Validation: Tuning the Graphical Lasso for Graph Recovery
• Information criteria for non-normalized models
• Causal Structural Learning Via Local Graphs
• Estimating Density Models with Truncation Boundaries using Score Matching
• Direct estimation and inference of differential Granger causality between two high-dimensional time series
• Interaction Models and Generalized Score Matching for Compositional Data