RepMLP: Re-parameterizing Convolutions into Fully-connected Layers for Image Recognition

Abstract

We propose RepMLP, a multi-layer-perceptron-style neural network building block for image recognition, which is composed of a series of fully-connected (FC) layers. Compared to convolutional layers, FC layers are more efficient, better at modeling the long-range dependencies and positional patterns, but worse at capturing the local structures, hence usually less favored for image recognition. We propose a structural re-parameterization technique that adds local prior into an FC to make it powerful for image recognition. Specifically, we construct convolutional layers inside a RepMLP during training and merge them into the FC for inference. On CIFAR, a simple pure-MLP model shows performance very close to CNN. By inserting RepMLP in traditional CNN, we improve ResNets by 1.8% accuracy on ImageNet, 2.9% for face recognition, and 2.3% mIoU on Cityscapes with lower FLOPs. Our intriguing findings highlight that combining the global representational capacity and positional perception of FC with the local prior of convolution can improve the performance of neural network with faster speed on both the tasks with translation invariance (e.g., semantic segmentation) and those with aligned images and positional patterns (e.g., face recognition). The code and models are available at https://github.com/DingXiaoH/RepMLP.

This was a work in progress. Latest version is arXiv:2112.11081 (accepted by CVPR 2022)

References in corpus (7)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• AS-MLP: An Axial Shifted MLP Architecture for Vision
• MEC: Memory-efficient Convolution for Deep Neural Network
• RepVGG: Making VGG-style ConvNets Great Again
• Pay Attention to MLPs

Cited by in corpus (6)

• XCiT: Cross-Covariance Image Transformers
• ViTAE: Vision Transformer Advanced by Exploring Intrinsic Inductive Bias
• Graph-MLP: Node Classification without Message Passing in Graph
• Pay Attention to MLPs
• Parameterization of Cross-Token Relations with Relative Positional Encoding for Vision MLP
• RepNAS: Searching for Efficient Re-parameterizing Blocks