Sliced Kernelized Stein Discrepancy
arXiv:2006.16531
Abstract
Kernelized Stein discrepancy (KSD), though being extensively used in goodness-of-fit tests and model learning, suffers from the curse-of-dimensionality. We address this issue by proposing the sliced Stein discrepancy and its scalable and kernelized variants, which employ kernel-based test functions defined on the optimal one-dimensional projections. When applied to goodness-of-fit tests, extensive experiments show the proposed discrepancy significantly outperforms KSD and various baselines in high dimensions. For model learning, we show its advantages over existing Stein discrepancy baselines by training independent component analysis models with different discrepancies. We further propose a novel particle inference method called sliced Stein variational gradient descent (S-SVGD) which alleviates the mode-collapse issue of SVGD in training variational autoencoders.
arXiv commentPublished at ICLR 2021
References in corpus (21)
- Adam: A Method for Stochastic Optimization
- Stochastic Backpropagation and Approximate Inference in Deep Generative Models
- Stochastic Gradient Hamiltonian Monte Carlo
- Demystifying MMD GANs
- Generative Models and Model Criticism via Optimized Maximum Mean Discrepancy
- Measuring Sample Quality with Stein's Method
- On integral probability metrics, ϕ-divergences and binary classification
- A Kernel Test of Goodness of Fit
- Learning Deep Kernels for Non-Parametric Two-Sample Tests
- Sliced Score Matching: A Scalable Approach to Density and Score Estimation
- Operator Variational Inference
- On gradient regularizers for MMD GANs
- A Linear-Time Kernel Goodness-of-Fit Test
- Learning the Stein Discrepancy for Training and Evaluating Energy-Based Models without Sampling
- Learning to Draw Samples with Amortized Stein Variational Gradient Descent
- Random Feature Stein Discrepancies
- Projected Stein Variational Gradient Descent
- VAE Learning via Stein Variational Gradient Descent
- Stein Neural Sampler
- Two Methods For Wild Variational Inference
- Kernelized Complete Conditional Stein Discrepancy