Banded Matrix Operators for Gaussian Markov Models in the Automatic Differentiation Era

Abstract

Banded matrices can be used as precision matrices in several models including linear state-space models, some Gaussian processes, and Gaussian Markov random fields. The aim of the paper is to make modern inference methods (such as variational inference or gradient-based sampling) available for Gaussian models with banded precision. We show that this can efficiently be achieved by equipping an automatic differentiation framework, such as TensorFlow or PyTorch, with some linear algebra operators dedicated to banded matrices. This paper studies the algorithmic aspects of the required operators, details their reverse-mode derivatives, and show that their complexity is linear in the number of observations.

Cited by in corpus (10)

• Variational Bayesian Approximation of Inverse Problems using Sparse Precision Matrices
• Gaussian variational approximation for high-dimensional state space models
• Fast Deep Mixtures of Gaussian Process Experts
• Pathwise Conditioning of Gaussian Processes
• Spatio-Temporal Variational Gaussian Processes
• Doubly Sparse Variational Gaussian Processes
• Matérn Gaussian Processes on Graphs
• Black-Box Inference for Non-Linear Latent Force Models
• Entropy-based adaptive Hamiltonian Monte Carlo
• On MCMC for variationally sparse Gaussian processes: A pseudo-marginal approach