Pathwise Derivatives Beyond the Reparameterization Trick

Abstract

We observe that gradients computed via the reparameterization trick are in direct correspondence with solutions of the transport equation in the formalism of optimal transport. We use this perspective to compute (approximate) pathwise gradients for probability distributions not directly amenable to the reparameterization trick: Gamma, Beta, and Dirichlet. We further observe that when the reparameterization trick is applied to the Cholesky-factorized multivariate Normal distribution, the resulting gradients are suboptimal in the sense of optimal transport. We derive the optimal gradients and show that they have reduced variance in a Gaussian Process regression task. We demonstrate with a variety of synthetic experiments and stochastic variational inference tasks that our pathwise gradients are competitive with other methods.

arXiv commentICML 2018

References in corpus (11)

• Auto-Encoding Variational Bayes
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• Gradient Estimation Using Stochastic Computation Graphs
• REBAR: Low-variance, unbiased gradient estimates for discrete latent variable models
• Implicit Reparameterization Gradients
• Variational Bayesian Inference with Stochastic Search
• Backpropagation through the Void: Optimizing control variates for black-box gradient estimation
• Automated Variational Inference in Probabilistic Programming
• Reducing Reparameterization Gradient Variance
• Stochastic gradient variational Bayes for gamma approximating distributions
• Stochastic Backpropagation through Mixture Density Distributions

Cited by in corpus (23)

• Normalizing Flows: An Introduction and Review of Current Methods
• Machine learning with neural networks
• Monte Carlo Gradient Estimation in Machine Learning
• Tighter risk certificates for neural networks
• All You Need is a Good Functional Prior for Bayesian Deep Learning
• ADEV: Sound Automatic Differentiation of Expected Values of Probabilistic Programs
• PAC-Bayes with Backprop
• Sparse Conditional Hidden Markov Model for Weakly Supervised Named Entity Recognition
• Learning Interpretable Deep Disentangled Neural Networks for Hyperspectral Unmixing
• Dirichlet Variational Autoencoder for Text Modeling
• f-Divergence Variational Inference
• Hierarchical Indian Buffet Neural Networks for Bayesian Continual Learning
• Differentiable Particle Filtering without Modifying the Forward Pass
• DynaNet: Neural Kalman Dynamical Model for Motion Estimation and Prediction
• DrNAS: Dirichlet Neural Architecture Search
• Variational Auto-encoder Based Bayesian Poisson Tensor Factorization for Sparse and Imbalanced Count Data
• Automatic Differentiation Variational Inference with Mixtures
• Hierarchical Variational Auto-Encoding for Unsupervised Domain Generalization
• Flexible mean field variational inference using mixtures of non-overlapping exponential families
• A New Distribution on the Simplex with Auto-Encoding Applications
• Bayesian Attention Modules
• Pathwise Derivatives for Multivariate Distributions
• Perturbative estimation of stochastic gradients