Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation

Abstract

Recently, Neural Architecture Search (NAS) has successfully identified neural network architectures that exceed human designed ones on large-scale image classification. In this paper, we study NAS for semantic image segmentation. Existing works often focus on searching the repeatable cell structure, while hand-designing the outer network structure that controls the spatial resolution changes. This choice simplifies the search space, but becomes increasingly problematic for dense image prediction which exhibits a lot more network level architectural variations. Therefore, we propose to search the network level structure in addition to the cell level structure, which forms a hierarchical architecture search space. We present a network level search space that includes many popular designs, and develop a formulation that allows efficient gradient-based architecture search (3 P100 GPU days on Cityscapes images). We demonstrate the effectiveness of the proposed method on the challenging Cityscapes, PASCAL VOC 2012, and ADE20K datasets. Auto-DeepLab, our architecture searched specifically for semantic image segmentation, attains state-of-the-art performance without any ImageNet pretraining.

arXiv commentTo appear in CVPR 2019 as oral. Code for Auto-DeepLab released at https://github.com/tensorflow/models/tree/master/research/deeplab

References in corpus (21)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Sequence to Sequence Learning with Neural Networks
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Rethinking Atrous Convolution for Semantic Image Segmentation
• LSTM: A Search Space Odyssey
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
• DARTS: Differentiable Architecture Search
• ParseNet: Looking Wider to See Better
• DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
• Large-Scale Evolution of Image Classifiers
• DropBlock: A regularization method for convolutional networks
• Neural Architecture Optimization
• Convolutional Feature Masking for Joint Object and Stuff Segmentation
• Designing Neural Network Architectures using Reinforcement Learning
• Searching for Efficient Multi-Scale Architectures for Dense Image Prediction
• Beyond Skip Connections: Top-Down Modulation for Object Detection
• Efficient Architecture Search by Network Transformation
• Stacked Deconvolutional Network for Semantic Segmentation
• MaskConnect: Connectivity Learning by Gradient Descent
• ExFuse: Enhancing Feature Fusion for Semantic Segmentation

Cited by in corpus (52)

• EfficientNetV2: Smaller Models and Faster Training
• Object Detection in 20 Years: A Survey
• Deep High-Resolution Representation Learning for Visual Recognition
• DARTS+: Improved Differentiable Architecture Search with Early Stopping
• Hierarchical Neural Architecture Search for Deep Stereo Matching
• A Survey on Neural Architecture Search
• DetNAS: Backbone Search for Object Detection
• Evaluating the Search Phase of Neural Architecture Search
• Polarized Self-Attention: Towards High-quality Pixel-wise Regression
• Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluation
• Gated-SCNN: Gated Shape CNNs for Semantic Segmentation
• Auto-Keras: An Efficient Neural Architecture Search System
• Efficient Neural Architecture Transformation Searchin Channel-Level for Object Detection
• Global Aggregation then Local Distribution in Fully Convolutional Networks
• AutoGAN: Neural Architecture Search for Generative Adversarial Networks
• Stabilizing DARTS with Amended Gradient Estimation on Architectural Parameters
• SPGNet: Semantic Prediction Guidance for Scene Parsing
• GFF: Gated Fully Fusion for Semantic Segmentation
• Sample-Efficient Neural Architecture Search by Learning Action Space
• Pedestrian Attribute Recognition: A Survey
• DDCNet: Deep Dilated Convolutional Neural Network for Dense Prediction
• Learning Graph Convolutional Network for Skeleton-based Human Action Recognition by Neural Searching
• How to Train Your Super-Net: An Analysis of Training Heuristics in Weight-Sharing NAS
• Deep Neural Network Fingerprinting by Conferrable Adversarial Examples
• Multinomial Distribution Learning for Effective Neural Architecture Search
• Trilevel Neural Architecture Search for Efficient Single Image Super-Resolution
• SM-NAS: Structural-to-Modular Neural Architecture Search for Object Detection
• Pose Neural Fabrics Search
• Neural Architecture Search for Deep Face Recognition
• IrisNet: Deep Learning for Automatic and Real-time Tongue Contour Tracking in Ultrasound Video Data using Peripheral Vision
• Graph-guided Architecture Search for Real-time Semantic Segmentation
• ObjectAug: Object-level Data Augmentation for Semantic Image Segmentation
• Towards Oracle Knowledge Distillation with Neural Architecture Search
• Video Action Recognition Via Neural Architecture Searching
• Modeling Neural Architecture Search Methods for Deep Networks
• Searching for Stage-wise Neural Graphs In the Limit
• Neural Architecture Refinement: A Practical Way for Avoiding Overfitting in NAS
• Rethinking Performance Estimation in Neural Architecture Search
• Neural Architecture Search for Gliomas Segmentation on Multimodal Magnetic Resonance Imaging
• AIO-P: Expanding Neural Performance Predictors Beyond Image Classification
• Joint-DetNAS: Upgrade Your Detector with NAS, Pruning and Dynamic Distillation
• A deep learning based multiscale approach to segment cancer area in liver whole slide image
• Progressive Neural Index Search for Database System
• ElixirNet: Relation-aware Network Architecture Adaptation for Medical Lesion Detection
• Towards Cardiac Intervention Assistance: Hardware-aware Neural Architecture Exploration for Real-Time 3D Cardiac Cine MRI Segmentation
• WAS-VTON: Warping Architecture Search for Virtual Try-on Network
• UXNet: Searching Multi-level Feature Aggregation for 3D Medical Image Segmentation
• Efficient Data-specific Model Search for Collaborative Filtering
• Searching Learning Strategy with Reinforcement Learning for 3D Medical Image Segmentation
• Neural Architecture Search for Image Super-Resolution Using Densely Constructed Search Space: DeCoNAS
• SparseMask: Differentiable Connectivity Learning for Dense Image Prediction
• LV-BERT: Exploiting Layer Variety for BERT