Gaussian RBF Centered Kernel Alignment (CKA) in the Large Bandwidth Limit
arXiv:2112.09305 · doi:10.1109/TPAMI.2022.3216518
Abstract
We prove that Centered Kernel Alignment (CKA) based on a Gaussian RBF kernel converges to linear CKA in the large-bandwidth limit. We show that convergence onset is sensitive to the geometry of the feature representations, and that representation eccentricity bounds the range of bandwidths for which Gaussian CKA behaves nonlinearly.
arXiv comment11 pages, 3 figures
References in corpus (6)
- Array Programming with NumPy
- Kernel methods in machine learning
- OpenML: networked science in machine learning
- Algorithms for Learning Kernels Based on Centered Alignment
- Do Wide and Deep Networks Learn the Same Things? Uncovering How Neural Network Representations Vary with Width and Depth
- Universality, Characteristic Kernels and RKHS Embedding of Measures