Efficient Neural Architecture Transformation Searchin Channel-Level for Object Detection

Abstract

Recently, Neural Architecture Search has achieved great success in large-scale image classification. In contrast, there have been limited works focusing on architecture search for object detection, mainly because the costly ImageNet pre-training is always required for detectors. Training from scratch, as a substitute, demands more epochs to converge and brings no computation saving. To overcome this obstacle, we introduce a practical neural architecture transformation search(NATS)algorithm for object detection in this paper. Instead of searching and constructing an entire network, NATS explores the architecture space on the base of existing network and reusing its weights. We propose a novel neural architecture search strategy in channel-level instead of path-level and devise a search space specially targeting at object detection. With the combination of these two designs, an architecture transformation scheme could be discovered to adapt a network designed for image classification to task of object detection. Since our method is gradient-based and only searches for a transformation scheme, the weights of models pretrained inImageNet could be utilized in both searching and retraining stage, which makes the whole process very efficient. The transformed network requires no extra parameters and FLOPs, and is friendly to hardware optimization, which is practical to use in real-time application. In experiments, we demonstrate the effectiveness of NATSon networks like ResNet and ResNeXt. Our transformed networks, combined with various detection frameworks, achieve significant improvements on the COCO dataset while keeping fast.

References in corpus (12)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Neural Architecture Search with Reinforcement Learning
• Understanding the Effective Receptive Field in Deep Convolutional Neural Networks
• Efficient Neural Architecture Search via Parameter Sharing
• Large-Scale Evolution of Image Classifiers
• Designing Neural Network Architectures using Reinforcement Learning
• Searching for Efficient Multi-Scale Architectures for Dense Image Prediction
• Efficient Architecture Search by Network Transformation
• Deformable ConvNets v2: More Deformable, Better Results
• Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation
• Rethinking ImageNet Pre-training

Cited by in corpus (8)

• A Comprehensive Survey of Neural Architecture Search: Challenges and Solutions
• Hit-Detector: Hierarchical Trinity Architecture Search for Object Detection
• AdversarialNAS: Adversarial Neural Architecture Search for GANs
• Feature Pyramid Grids
• AlphaGAN: Fully Differentiable Architecture Search for Generative Adversarial Networks
• PV-NAS: Practical Neural Architecture Search for Video Recognition
• Cheaper Pre-training Lunch: An Efficient Paradigm for Object Detection
• Representation Sharing for Fast Object Detector Search and Beyond