Large-scale Log-determinant Computation through Stochastic Chebyshev Expansions

Abstract

Logarithms of determinants of large positive definite matrices appear ubiquitously in machine learning applications including Gaussian graphical and Gaussian process models, partition functions of discrete graphical models, minimum-volume ellipsoids, metric learning and kernel learning. Log-determinant computation involves the Cholesky decomposition at the cost cubic in the number of variables, i.e., the matrix dimension, which makes it prohibitive for large-scale applications. We propose a linear-time randomized algorithm to approximate log-determinants for very large-scale positive definite and general non-singular matrices using a stochastic trace approximation, called the Hutchinson method, coupled with Chebyshev polynomial expansions that both rely on efficient matrix-vector multiplications. We establish rigorous additive and multiplicative approximation error bounds depending on the condition number of the input matrix. In our experiments, the proposed algorithm can provide very high accuracy solutions at orders of magnitude faster time than the Cholesky decomposition and Schur completion, and enables us to compute log-determinants of matrices involving tens of millions of variables.

References in corpus (3)

• Robust inversion via semistochastic dimensionality reduction
• Efficient Exact Inference in Planar Ising Models
• Stochastic determination of matrix determinants

Cited by in corpus (12)

• Geometric variational inference
• MEMe: An Accurate Maximum Entropy Method for Efficient Approximations in Large-Scale Machine Learning
• Algorithms for the computation of the matrix logarithm based on the double exponential formula
• Exponential Family Estimation via Adversarial Dynamics Embedding
• Large-Scale Wasserstein Gradient Flows
• Schatten Norms in Matrix Streams: Hello Sparsity, Goodbye Dimension
• Quantum algorithms for spectral sums
• Rectangular Flows for Manifold Learning
• Two equalities expressing the determinant of a matrix in terms of expectations over matrix-vector products
• A determinant-free method to simulate the parameters of large Gaussian fields
• Bayesian Variable Selection for Single Index Logistic Model
• Randomized block Krylov space methods for trace and log-determinant estimators