Fast-SCNN: Fast Semantic Segmentation Network

Abstract

The encoder-decoder framework is state-of-the-art for offline semantic image segmentation. Since the rise in autonomous systems, real-time computation is increasingly desirable. In this paper, we introduce fast segmentation convolutional neural network (Fast-SCNN), an above real-time semantic segmentation model on high resolution image data (1024x2048px) suited to efficient computation on embedded devices with low memory. Building on existing two-branch methods for fast segmentation, we introduce our `learning to downsample' module which computes low-level features for multiple resolution branches simultaneously. Our network combines spatial detail at high resolution with deep features extracted at lower resolution, yielding an accuracy of 68.0% mean intersection over union at 123.5 frames per second on Cityscapes. We also show that large scale pre-training is unnecessary. We thoroughly validate our metric in experiments with ImageNet pre-training and the coarse labeled data of Cityscapes. Finally, we show even faster computation with competitive results on subsampled inputs, without any network modifications.

References in corpus (13)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
• Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding
• ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation
• The Cityscapes Dataset for Semantic Urban Scene Understanding
• Binarized Neural Networks
• Fully Convolutional Networks for Semantic Segmentation
• Rich feature hierarchies for accurate object detection and semantic segmentation
• Xception: Deep Learning with Depthwise Separable Convolutions
• Pyramid Scene Parsing Network
• ContextNet: Exploring Context and Detail for Semantic Segmentation in Real-time
• Guided Upsampling Network for Real-Time Semantic Segmentation

Cited by in corpus (36)

• Rethinking Pre-training and Self-training
• nnFormer: Interleaved Transformer for Volumetric Segmentation
• Deep Dual-resolution Networks for Real-time and Accurate Semantic Segmentation of Road Scenes
• BiSeNet V2: Bilateral Network with Guided Aggregation for Real-time Semantic Segmentation
• FasterSeg: Searching for Faster Real-time Semantic Segmentation
• SalsaNext: Fast, Uncertainty-aware Semantic Segmentation of LiDAR Point Clouds for Autonomous Driving
• Towards Goal-Oriented Semantic Signal Processing: Applications and Future Challenges
• From SLAM to Situational Awareness: Challenges and Survey
• PaddleSeg: A High-Efficient Development Toolkit for Image Segmentation
• Real-Time Semantic Segmentation via Multiply Spatial Fusion Network
• Improving Semantic Segmentation via Self-Training
• Segmenting Transparent Object in the Wild with Transformer
• ExtremeC3Net: Extreme Lightweight Portrait Segmentation Networks using Advanced C3-modules
• Efficient Visual Recognition with Deep Neural Networks: A Survey on Recent Advances and New Directions
• Transferable Universal Adversarial Perturbations Using Generative Models
• Recurrent U-Net for Resource-Constrained Segmentation
• Segmenting Transparent Objects in the Wild
• A Survey on Deep Learning Methods for Semantic Image Segmentation in Real-Time
• MSCFNet: A Lightweight Network With Multi-Scale Context Fusion for Real-Time Semantic Segmentation
• Transformer Meets Convolution: A Bilateral Awareness Network for Semantic Segmentation of Very Fine Resolution Urban Scene Images
• A Three-Stage Self-Training Framework for Semi-Supervised Semantic Segmentation
• Multichannel Orthogonal Transform-Based Perceptron Layers for Efficient ResNets
• On Benchmarking Iris Recognition within a Head-mounted Display for AR/VR Application
• FDDWNet: A Lightweight Convolutional Neural Network for Real-time Sementic Segmentation
• FarSee-Net: Real-Time Semantic Segmentation by Efficient Multi-scale Context Aggregation and Feature Space Super-resolution
• GSVNet: Guided Spatially-Varying Convolution for Fast Semantic Segmentation on Video
• SINet: Extreme Lightweight Portrait Segmentation Networks with Spatial Squeeze Modules and Information Blocking Decoder
• BABO: Background Activation Black-Out for Efficient Object Detection
• Fast Walsh-Hadamard Transform and Smooth-Thresholding Based Binary Layers in Deep Neural Networks
• Feature boosting with efficient attention for scene parsing
• Using constraint structure and an improved object detection network to detect the 12^{th} Vertebra from CT images with a limited field of view for image-guided radiotherapy
• Centimeter-Wave Free-Space Time-of-Flight Imaging
• Dense Dual-Path Network for Real-time Semantic Segmentation
• Intra-class Patch Swap for Self-Distillation
• Self-Improving Semantic Perception for Indoor Localisation
• AASeg: Attention Aware Network for Real Time Semantic Segmentation