Learning Manifold Implicitly via Explicit Heat-Kernel Learning
arXiv:2010.01761
Abstract
Manifold learning is a fundamental problem in machine learning with numerous applications. Most of the existing methods directly learn the low-dimensional embedding of the data in some high-dimensional space, and usually lack the flexibility of being directly applicable to down-stream applications. In this paper, we propose the concept of implicit manifold learning, where manifold information is implicitly obtained by learning the associated heat kernel. A heat kernel is the solution of the corresponding heat equation, which describes how "heat" transfers on the manifold, thus containing ample geometric information of the manifold. We provide both practical algorithm and theoretical analysis of our framework. The learned heat kernel can be applied to various kernel-based machine learning models, including deep generative models (DGM) for data generation and Stein Variational Gradient Descent for Bayesian inference. Extensive experiments show that our framework can achieve state-of-the-art results compared to existing methods for the two tasks.
arXiv commentAccepted by NeurIPS 2020. Some typos have been corrected
References in corpus (14)
- Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
- GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
- Spectral Normalization for Generative Adversarial Networks
- MMD GAN: Towards Deeper Understanding of Moment Matching Network
- Geodesics in Heat
- Demystifying MMD GANs
- On gradient regularizers for MMD GANs
- Improving MMD-GAN Training with Repulsive Loss Function
- Stein Variational Gradient Descent With Matrix-Valued Kernels
- Implicit Kernel Learning
- Latent Space Oddity: on the Curvature of Deep Generative Models
- The Riemannian Geometry of Deep Generative Models
- Variational Autoencoders with Riemannian Brownian Motion Priors
- KernelNet: A Data-Dependent Kernel Parameterization for Deep Generative Modeling