Cyclic orthogonal convolutions for long-range integration of features

Abstract

In Convolutional Neural Networks (CNNs) information flows across a small neighbourhood of each pixel of an image, preventing long-range integration of features before reaching deep layers in the network. We propose a novel architecture that allows flexible information flow between features $z$ and locations $(x,y)$ across the entire image with a small number of layers. This architecture uses a cycle of three orthogonal convolutions, not only in $(x,y)$ coordinates, but also in $(x,z)$ and $(y,z)$ coordinates. We stack a sequence of such cycles to obtain our deep network, named CycleNet. As this only requires a permutation of the axes of a standard convolution, its performance can be directly compared to a CNN. Our model obtains competitive results at image classification on CIFAR-10 and ImageNet datasets, when compared to CNNs of similar size. We hypothesise that long-range integration favours recognition of objects by shape rather than texture, and we show that CycleNet transfers better than CNNs to stylised images. On the Pathfinder challenge, where integration of distant features is crucial, CycleNet outperforms CNNs by a large margin. We also show that even when employing a small convolutional kernel, the size of receptive fields of CycleNet reaches its maximum after one cycle, while conventional CNNs require a large number of layers.

arXiv comment11 pages, 5 figures

References in corpus (11)

• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
• Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
• Multi-Scale Context Aggregation by Dilated Convolutions
• Tensorizing Neural Networks
• Horovod: fast and easy distributed deep learning in TensorFlow
• Deformable Convolutional Networks
• Stand-Alone Self-Attention in Vision Models
• Non-local Neural Networks
• Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNet
• Taxonomy and Evaluation of Structured Compression of Convolutional Neural Networks