GhostNets on Heterogeneous Devices via Cheap Operations

Abstract

Deploying convolutional neural networks (CNNs) on mobile devices is difficult due to the limited memory and computation resources. We aim to design efficient neural networks for heterogeneous devices including CPU and GPU, by exploiting the redundancy in feature maps, which has rarely been investigated in neural architecture design. For CPU-like devices, we propose a novel CPU-efficient Ghost (C-Ghost) module to generate more feature maps from cheap operations. Based on a set of intrinsic feature maps, we apply a series of linear transformations with cheap cost to generate many ghost feature maps that could fully reveal information underlying intrinsic features. The proposed C-Ghost module can be taken as a plug-and-play component to upgrade existing convolutional neural networks. C-Ghost bottlenecks are designed to stack C-Ghost modules, and then the lightweight C-GhostNet can be easily established. We further consider the efficient networks for GPU devices. Without involving too many GPU-inefficient operations (e.g.,, depth-wise convolution) in a building stage, we propose to utilize the stage-wise feature redundancy to formulate GPU-efficient Ghost (G-Ghost) stage structure. The features in a stage are split into two parts where the first part is processed using the original block with fewer output channels for generating intrinsic features, and the other are generated using cheap operations by exploiting stage-wise redundancy. Experiments conducted on benchmarks demonstrate the effectiveness of the proposed C-Ghost module and the G-Ghost stage. C-GhostNet and G-GhostNet can achieve the optimal trade-off of accuracy and latency for CPU and GPU, respectively. Code is available at https://github.com/huawei-noah/CV-Backbones.

arXiv commentAccepted by IJCV 2022. Extension of GhostNet CVPR2020 paper (arXiv:1911.11907). arXiv admin note: substantial text overlap with arXiv:1911.11907

References in corpus (20)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
• Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
• Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding
• Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
• AMC: AutoML for Model Compression and Acceleration on Mobile Devices
• MMDetection: Open MMLab Detection Toolbox and Benchmark
• Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels
• ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware
• FasterSeg: Searching for Faster Real-time Semantic Segmentation
• Model Compression with Adversarial Robustness: A Unified Optimization Framework
• Searching for Low-Bit Weights in Quantized Neural Networks
• Training Binary Neural Networks through Learning with Noisy Supervision
• Provable Filter Pruning for Efficient Neural Networks
• Constructing Fast Network through Deconstruction of Convolution
• Learning Versatile Convolution Filters for Efficient Visual Recognition