Convolutional Spectral Kernel Learning

Abstract

Recently, non-stationary spectral kernels have drawn much attention, owing to its powerful feature representation ability in revealing long-range correlations and input-dependent characteristics. However, non-stationary spectral kernels are still shallow models, thus they are deficient to learn both hierarchical features and local interdependence. In this paper, to obtain hierarchical and local knowledge, we build an interpretable convolutional spectral kernel network (\texttt{CSKN}) based on the inverse Fourier transform, where we introduce deep architectures and convolutional filters into non-stationary spectral kernel representations. Moreover, based on Rademacher complexity, we derive the generalization error bounds and introduce two regularizers to improve the performance. Combining the regularizers and recent advancements on random initialization, we finally complete the learning framework of \texttt{CSKN}. Extensive experiments results on real-world datasets validate the effectiveness of the learning framework and coincide with our theoretical findings.

References in corpus (10)

• Adam: A Method for Stochastic Optimization
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Neural Tangent Kernel: Convergence and Generalization in Neural Networks
• Generalization Properties of Learning with Random Features
• To understand deep learning we need to understand kernel learning
• Non-Stationary Spectral Kernels
• A Kernel Perspective for Regularizing Deep Neural Networks
• Generalized Spectral Kernels
• Automated Spectral Kernel Learning
• Stacked Kernel Network