HR-NAS: Searching Efficient High-Resolution Neural Architectures with Lightweight Transformers

Abstract

High-resolution representations (HR) are essential for dense prediction tasks such as segmentation, detection, and pose estimation. Learning HR representations is typically ignored in previous Neural Architecture Search (NAS) methods that focus on image classification. This work proposes a novel NAS method, called HR-NAS, which is able to find efficient and accurate networks for different tasks, by effectively encoding multiscale contextual information while maintaining high-resolution representations. In HR-NAS, we renovate the NAS search space as well as its searching strategy. To better encode multiscale image contexts in the search space of HR-NAS, we first carefully design a lightweight transformer, whose computational complexity can be dynamically changed with respect to different objective functions and computation budgets. To maintain high-resolution representations of the learned networks, HR-NAS adopts a multi-branch architecture that provides convolutional encoding of multiple feature resolutions, inspired by HRNet. Last, we proposed an efficient fine-grained search strategy to train HR-NAS, which effectively explores the search space, and finds optimal architectures given various tasks and computation resources. HR-NAS is capable of achieving state-of-the-art trade-offs between performance and FLOPs for three dense prediction tasks and an image classification task, given only small computational budgets. For example, HR-NAS surpasses SqueezeNAS that is specially designed for semantic segmentation while improving efficiency by 45.9%. Code is available at https://github.com/dingmyu/HR-NAS

arXiv commentAccepted by CVPR 2021 (Oral)

References in corpus (26)

• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• mixup: Beyond Empirical Risk Minimization
• Language Models are Few-Shot Learners
• Improved Regularization of Convolutional Neural Networks with Cutout
• DARTS: Differentiable Architecture Search
• ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation
• Linformer: Self-Attention with Linear Complexity
• Searching for Activation Functions
• AutoAugment: Learning Augmentation Policies from Data
• Large-Scale Evolution of Image Classifiers
• Generating Long Sequences with Sparse Transformers
• Visual Transformers: Token-based Image Representation and Processing for Computer Vision
• Searching for Efficient Multi-Scale Architectures for Dense Image Prediction
• MixConv: Mixed Depthwise Convolutional Kernels
• SNAS: Stochastic Neural Architecture Search
• DARTS+: Improved Differentiable Architecture Search with Early Stopping
• The Evolved Transformer
• FastBERT: a Self-distilling BERT with Adaptive Inference Time
• RC-DARTS: Resource Constrained Differentiable Architecture Search
• Meta Architecture Search
• An efficient solution for semantic segmentation: ShuffleNet V2 with atrous separable convolutions
• DCNAS: Densely Connected Neural Architecture Search for Semantic Image Segmentation
• C3: Concentrated-Comprehensive Convolution and its application to semantic segmentation
• AutoPose: Searching Multi-Scale Branch Aggregation for Pose Estimation
• Pose Neural Fabrics Search
• Learning Versatile Neural Architectures by Propagating Network Codes