Differentiation of the Cholesky decomposition
arXiv:1602.07527
Abstract
We review strategies for differentiating matrix-based computations, and derive symbolic and algorithmic update rules for differentiating expressions containing the Cholesky decomposition. We recommend new `blocked' algorithms, based on differentiating the Cholesky algorithm DPOTRF in the LAPACK library, which uses `Level 3' matrix-matrix operations from BLAS, and so is cache-friendly and easy to parallelize. For large matrices, the resulting algorithms are the fastest way to compute Cholesky derivatives, and are an order of magnitude faster than the algorithms in common usage. In some computing environments, symbolically-derived updates are faster for small matrices than those based on differentiating Cholesky algorithms. The symbolic and algorithmic approaches can be combined to get the best of both worlds.
arXiv comment18 pages, including 7 pages of code listings
References in corpus (1)
Cited by in corpus (8)
- Supervised Fitting of Geometric Primitives to 3D Point Clouds
- On the Computation of Complex-valued Gradients with Application to Statistically Optimum Beamforming
- Auto-Differentiating Linear Algebra
- Bayesian Optimization of Risk Measures
- What is the gradient of a scalar function of a symmetric matrix ?
- Depth Completion via Deep Basis Fitting
- GPU-based Parallel Computation Support for Stan
- Eigendecomposition-Free Training of Deep Networks for Linear Least-Square Problems