Real-Time Semantic Segmentation via Auto Depth, Downsampling Joint Decision and Feature Aggregation

Abstract

To satisfy the stringent requirements on computational resources in the field of real-time semantic segmentation, most approaches focus on the hand-crafted design of light-weight segmentation networks. Recently, Neural Architecture Search (NAS) has been used to search for the optimal building blocks of networks automatically, but the network depth, downsampling strategy, and feature aggregation way are still set in advance by trial and error. In this paper, we propose a joint search framework, called AutoRTNet, to automate the design of these strategies. Specifically, we propose hyper-cells to jointly decide the network depth and downsampling strategy, and an aggregation cell to achieve automatic multi-scale feature aggregation. Experimental results show that AutoRTNet achieves 73.9% mIoU on the Cityscapes test set and 110.0 FPS on an NVIDIA TitanXP GPU card with 768x1536 input images.

arXiv commentsubmitted to IJCV

References in corpus (17)

• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Rethinking Atrous Convolution for Semantic Image Segmentation
• Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
• Neural Architecture Search with Reinforcement Learning
• Categorical Reparameterization with Gumbel-Softmax
• DARTS: Differentiable Architecture Search
• The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
• ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation
• Understanding the Effective Receptive Field in Deep Convolutional Neural Networks
• Searching for Efficient Multi-Scale Architectures for Dense Image Prediction
• ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware
• Random Search and Reproducibility for Neural Architecture Search
• Pyramid Attention Network for Semantic Segmentation
• Mixed Precision Quantization of ConvNets via Differentiable Neural Architecture Search
• DABNet: Depth-wise Asymmetric Bottleneck for Real-time Semantic Segmentation
• High-performance Semantic Segmentation Using Very Deep Fully Convolutional Networks
• Real-time Semantic Image Segmentation via Spatial Sparsity