Learning deep kernels for exponential family densities
arXiv:1811.08357
Abstract
The kernel exponential family is a rich class of distributions, which can be fit efficiently and with statistical guarantees by score matching. Being required to choose a priori a simple kernel such as the Gaussian, however, limits its practical applicability. We provide a scheme for learning a kernel parameterized by a deep network, which can find complex location-dependent local features of the data geometry. This gives a very rich class of density models, capable of fitting complex structures on moderate-dimensional problems. Compared to deep density models fit via maximum likelihood, our approach provides a complementary set of strengths and tradeoffs: in empirical studies, the former can yield higher likelihoods, whereas the latter gives better estimates of the gradient of the log density, the score, which describes the distribution's shape.
References in corpus (13)
- Pixel Recurrent Neural Networks
- Variational Inference with Normalizing Flows
- MMD GAN: Towards Deeper Understanding of Moment Matching Network
- The Cramer Distance as a Solution to Biased Wasserstein Gradients
- A Kernel Test of Goodness of Fit
- Density Estimation in Infinite Dimensional Exponential Families
- Interpretation and Generalization of Score Matching
- On gradient regularizers for MMD GANs
- Semi-supervised Deep Kernel Learning: Regression with Unlabeled Data by Minimizing Predictive Variance
- A Linear-Time Kernel Goodness-of-Fit Test
- Deep Energy Estimator Networks
- Gradient-free Hamiltonian Monte Carlo with Efficient Kernel Exponential Families
- Informative Features for Model Comparison
Cited by in corpus (6)
- On the representation and learning of monotone triangular transport maps
- Minimum Stein Discrepancy Estimators
- Batch Stationary Distribution Estimation
- Highly Accurate Real-space Electron Densities with Neural Networks
- Noise Contrastive Meta-Learning for Conditional Density Estimation using Kernel Mean Embeddings
- Kernel Mean Matching for Content Addressability of GANs