Integrating Large Circular Kernels into CNNs through Neural Architecture Search

Abstract

The square kernel is a standard unit for contemporary CNNs, as it fits well on the tensor computation for convolution operation. However, the retinal ganglion cells in the biological visual system have approximately concentric receptive fields. Motivated by this observation, we propose to use circular kernel with a concentric and isotropic receptive field as an option for the convolution operation. We first propose a simple yet efficient implementation of the convolution using circular kernels, and empirically show the significant advantages of large circular kernels over the counterpart square kernels. We then expand the operation space of several typical Neural Architecture Search (NAS) methods with the convolutions of large circular kernels. The searched new neural architectures do contain large circular kernels and outperform the original searched models considerably. Our additional analysis also reveals that large circular kernels could help the model to be more robust to the rotated or sheared images due to their better rotation invariance. Our work shows the potential of designing new convolutional kernels for CNNs, bringing up the prospect of expanding the search space of NAS with new variants of convolutions.

20 pages, 10 figures, submitted to a conference

References in corpus (9)

• Neural Architecture Search with Reinforcement Learning
• Improved Regularization of Convolutional Neural Networks with Cutout
• Understanding the Effective Receptive Field in Deep Convolutional Neural Networks
• Designing Neural Network Architectures using Reinforcement Learning
• A Comprehensive Survey of Neural Architecture Search: Challenges and Solutions
• BayesNAS: A Bayesian Approach for Neural Architecture Search
• AtomNAS: Fine-Grained End-to-End Neural Architecture Search
• Deformable Kernels: Adapting Effective Receptive Fields for Object Deformation
• Evolving Search Space for Neural Architecture Search