SM-NAS: Structural-to-Modular Neural Architecture Search for Object Detection

Abstract

The state-of-the-art object detection method is complicated with various modules such as backbone, feature fusion neck, RPN and RCNN head, where each module may have different designs and structures. How to leverage the computational cost and accuracy trade-off for the structural combination as well as the modular selection of multiple modules? Neural architecture search (NAS) has shown great potential in finding an optimal solution. Existing NAS works for object detection only focus on searching better design of a single module such as backbone or feature fusion neck, while neglecting the balance of the whole system. In this paper, we present a two-stage coarse-to-fine searching strategy named Structural-to-Modular NAS (SM-NAS) for searching a GPU-friendly design of both an efficient combination of modules and better modular-level architecture for object detection. Specifically, Structural-level searching stage first aims to find an efficient combination of different modules; Modular-level searching stage then evolves each specific module and pushes the Pareto front forward to a faster task-specific network. We consider a multi-objective search where the search space covers many popular designs of detection methods. We directly search a detection backbone without pre-trained models or any proxy task by exploring a fast training from scratch strategy. The resulting architectures dominate state-of-the-art object detection systems in both inference time and accuracy and demonstrate the effectiveness on multiple detection datasets, e.g. halving the inference time with additional 1% mAP improvement compared to FPN and reaching 46% mAP with the similar inference time of MaskRCNN.

Accepted by AAAI 2020

References in corpus (19)

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• YOLOv3: An Incremental Improvement
• EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
• R-FCN: Object Detection via Region-based Fully Convolutional Networks
• DSSD : Deconvolutional Single Shot Detector
• DARTS: Differentiable Architecture Search
• Large-Scale Evolution of Image Classifiers
• Searching for Efficient Multi-Scale Architectures for Dense Image Prediction
• Efficient Architecture Search by Network Transformation
• DetNet: A Backbone network for Object Detection
• Hierarchical Representations for Efficient Architecture Search
• DetNAS: Backbone Search for Object Detection
• Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation
• Rethinking ImageNet Pre-training
• Scale-Aware Trident Networks for Object Detection
• Revisiting Feature Alignment for One-stage Object Detection
• ThunderNet: Towards Real-time Generic Object Detection
• NAS-FCOS: Fast Neural Architecture Search for Object Detection
• ScratchDet: Training Single-Shot Object Detectors from Scratch

Cited by in corpus (5)

• Auto-Panoptic: Cooperative Multi-Component Architecture Search for Panoptic Segmentation
• TransNAS-Bench-101: Improving Transferability and Generalizability of Cross-Task Neural Architecture Search
• CATCH: Context-based Meta Reinforcement Learning for Transferrable Architecture Search
• Joint-DetNAS: Upgrade Your Detector with NAS, Pruning and Dynamic Distillation
• Representation Sharing for Fast Object Detector Search and Beyond