Neural Tangent Kernel Maximum Mean Discrepancy
arXiv:2106.03227
Abstract
We present a novel neural network Maximum Mean Discrepancy (MMD) statistic by identifying a new connection between neural tangent kernel (NTK) and MMD. This connection enables us to develop a computationally efficient and memory-efficient approach to compute the MMD statistic and perform NTK based two-sample tests towards addressing the long-standing challenge of memory and computational complexity of the MMD statistic, which is essential for online implementation to assimilating new samples. Theoretically, such a connection allows us to understand the NTK test statistic properties, such as the Type-I error and testing power for performing the two-sample test, by adapting existing theories for kernel MMD. Numerical experiments on synthetic and real-world datasets validate the theory and demonstrate the effectiveness of the proposed NTK-MMD statistic.
References in corpus (6)
- Generative Moment Matching Networks
- On integral probability metrics, ϕ-divergences and binary classification
- Enhanced Convolutional Neural Tangent Kernels
- Fast Two-Sample Testing with Analytic Representations of Probability Measures
- Limitations of Lazy Training of Two-layers Neural Networks
- Spectral Analysis of the Neural Tangent Kernel for Deep Residual Networks