Projected Stein Variational Newton: A Fast and Scalable Bayesian Inference Method in High Dimensions

Abstract

We propose a fast and scalable variational method for Bayesian inference in high-dimensional parameter space, which we call projected Stein variational Newton (pSVN) method. We exploit the intrinsic low-dimensional geometric structure of the posterior distribution in the high-dimensional parameter space via its Hessian (of the log posterior) operator and perform a parallel update of the parameter samples projected into a low-dimensional subspace by an SVN method. The subspace is adaptively constructed using the eigenvectors of the averaged Hessian at the current samples. We demonstrate fast convergence of the proposed method and its scalability with respect to the number of parameters, samples, and processor cores.

Cited by in corpus (15)

• Optimal design of acoustic metamaterial cloaks under uncertainty
• On the geometry of Stein variational gradient descent
• Solving Bayesian Inverse Problems via Variational Autoencoders
• Bayesian inference of heterogeneous epidemic models: Application to COVID-19 spread accounting for long-term care facilities
• Derivative-Informed Projected Neural Networks for High-Dimensional Parametric Maps Governed by PDEs
• Stein variational gradient descent with local approximations
• Enforcing Boundary Conditions on Physical Fields in Bayesian Inversion
• A fast and scalable computational framework for large-scale and high-dimensional Bayesian optimal experimental design
• Stein variational gradient descent on infinite-dimensional space and applications to statistical inverse problems
• Information Newton's flow: second-order optimization method in probability space
• Taylor approximation for chance constrained optimization problems governed by partial differential equations with high-dimensional random parameters
• Sampling with Mirrored Stein Operators
• Low Rank Saddle Free Newton: A Scalable Method for Stochastic Nonconvex Optimization
• Active Slices for Sliced Stein Discrepancy
• Multilevel Stein variational gradient descent with applications to Bayesian inverse problems