Spectral Representations for Convolutional Neural Networks

Abstract

Discrete Fourier transforms provide a significant speedup in the computation of convolutions in deep learning. In this work, we demonstrate that, beyond its advantages for efficient computation, the spectral domain also provides a powerful representation in which to model and train convolutional neural networks (CNNs). We employ spectral representations to introduce a number of innovations to CNN design. First, we propose spectral pooling, which performs dimensionality reduction by truncating the representation in the frequency domain. This approach preserves considerably more information per parameter than other pooling strategies and enables flexibility in the choice of pooling output dimensionality. This representation also enables a new form of stochastic regularization by randomized modification of resolution. We show that these methods achieve competitive results on classification and approximation tasks, without using any dropout or max-pooling. Finally, we demonstrate the effectiveness of complex-coefficient spectral parameterization of convolutional filters. While this leaves the underlying model unchanged, it results in a representation that greatly facilitates optimization. We observe on a variety of popular CNN configurations that this leads to significantly faster convergence during training.

References in corpus (4)

• Practical Bayesian Optimization of Machine Learning Algorithms
• Deeply-Supervised Nets
• Scalable Bayesian Optimization Using Deep Neural Networks
• Learning Ordered Representations with Nested Dropout

Cited by in corpus (23)

• Object Detection in 20 Years: A Survey
• Deep Roots: Improving CNN Efficiency with Hierarchical Filter Groups
• Photonic convolutional neural networks using integrated diffractive optics
• First and Second Order Methods for Online Convolutional Dictionary Learning
• Small-brain neural networks rapidly solve inverse problems with vortex Fourier encoders
• Intriguing Findings of Frequency Selection for Image Deblurring
• CHS-Net: A Deep learning approach for hierarchical segmentation of COVID-19 infected CT images
• Resolution learning in deep convolutional networks using scale-space theory
• Acceleration of Convolutional Neural Network Using FFT-Based Split Convolutions
• Compressing the Input for CNNs with the First-Order Scattering Transform
• Deep Learning in the Wavelet Domain
• Can we integrate spatial verification methods into neural-network loss functions for atmospheric science?
• LiftPool: Bidirectional ConvNet Pooling
• Faster Convergence in Deep-Predictive-Coding Networks to Learn Deeper Representations
• QFCNN: Quantum Fourier Convolutional Neural Network
• Frequency Domain Convolutional Neural Network: Accelerated CNN for Large Diabetic Retinopathy Image Classification
• Image classification in frequency domain with 2SReLU: a second harmonics superposition activation function
• Universal Pooling -- A New Pooling Method for Convolutional Neural Networks
• Fast Partial Fourier Transform
• Hyperparameter Transfer Learning with Adaptive Complexity
• Towards Diverse Perspective Learning with Selection over Multiple Temporal Poolings
• Temporal Pattern Attention for Multivariate Time Series Forecasting
• EZCrop: Energy-Zoned Channels for Robust Output Pruning