FBNetV5: Neural Architecture Search for Multiple Tasks in One Run

Abstract

Neural Architecture Search (NAS) has been widely adopted to design accurate and efficient image classification models. However, applying NAS to a new computer vision task still requires a huge amount of effort. This is because 1) previous NAS research has been over-prioritized on image classification while largely ignoring other tasks; 2) many NAS works focus on optimizing task-specific components that cannot be favorably transferred to other tasks; and 3) existing NAS methods are typically designed to be "proxyless" and require significant effort to be integrated with each new task's training pipelines. To tackle these challenges, we propose FBNetV5, a NAS framework that can search for neural architectures for a variety of vision tasks with much reduced computational cost and human effort. Specifically, we design 1) a search space that is simple yet inclusive and transferable; 2) a multitask search process that is disentangled with target tasks' training pipeline; and 3) an algorithm to simultaneously search for architectures for multiple tasks with a computational cost agnostic to the number of tasks. We evaluate the proposed FBNetV5 targeting three fundamental vision tasks -- image classification, object detection, and semantic segmentation. Models searched by FBNetV5 in a single run of search have outperformed the previous stateof-the-art in all the three tasks: image classification (e.g., +1.3% ImageNet top-1 accuracy under the same FLOPs as compared to FBNetV3), semantic segmentation (e.g., +1.8% higher ADE20K val. mIoU than SegFormer with 3.6x fewer FLOPs), and object detection (e.g., +1.1% COCO val. mAP with 1.2x fewer FLOPs as compared to YOLOX).

References in corpus (16)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Neural Architecture Search with Reinforcement Learning
• YOLOX: Exceeding YOLO Series in 2021
• The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
• SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers
• Per-Pixel Classification is Not All You Need for Semantic Segmentation
• Probabilistic two-stage detection
• FasterSeg: Searching for Faster Real-time Semantic Segmentation
• LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inference
• NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search
• Probabilistic Neural Architecture Search
• Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions
• MobileDets: Searching for Object Detection Architectures for Mobile Accelerators
• Progressive DARTS: Bridging the Optimization Gap for NAS in the Wild
• ScaleNAS: One-Shot Learning of Scale-Aware Representations for Visual Recognition
• AlphaNet: Improved Training of Supernets with Alpha-Divergence