ContextNet: Exploring Context and Detail for Semantic Segmentation in Real-time

Abstract

Modern deep learning architectures produce highly accurate results on many challenging semantic segmentation datasets. State-of-the-art methods are, however, not directly transferable to real-time applications or embedded devices, since naive adaptation of such systems to reduce computational cost (speed, memory and energy) causes a significant drop in accuracy. We propose ContextNet, a new deep neural network architecture which builds on factorized convolution, network compression and pyramid representation to produce competitive semantic segmentation in real-time with low memory requirement. ContextNet combines a deep network branch at low resolution that captures global context information efficiently with a shallow branch that focuses on high-resolution segmentation details. We analyse our network in a thorough ablation study and present results on the Cityscapes dataset, achieving 66.1% accuracy at 18.3 frames per second at full (1024x2048) resolution (41.9 fps with pipelined computations for streamed data).

arXiv commentPublished as a conference paper at British Machine Vision Conference (BMVC), 2018

References in corpus (7)

• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation
• The Cityscapes Dataset for Semantic Urban Scene Understanding
• Binarized Neural Networks
• Xception: Deep Learning with Depthwise Separable Convolutions
• ICNet for Real-Time Semantic Segmentation on High-Resolution Images
• Laplacian Pyramid Reconstruction and Refinement for Semantic Segmentation

Cited by in corpus (29)

• SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers
• Fast-SCNN: Fast Semantic Segmentation Network
• nnFormer: Interleaved Transformer for Volumetric Segmentation
• Landslide4Sense: Reference Benchmark Data and Deep Learning Models for Landslide Detection
• DABNet: Depth-wise Asymmetric Bottleneck for Real-time Semantic Segmentation
• Gen-LaneNet: A Generalized and Scalable Approach for 3D Lane Detection
• Universal Adversarial Examples in Remote Sensing: Methodology and Benchmark
• P2AT: Pyramid Pooling Axial Transformer for Real-time Semantic Segmentation
• Efficient Dense Modules of Asymmetric Convolution for Real-Time Semantic Segmentation
• Segmenting Transparent Object in the Wild with Transformer
• Backdoor Attacks for Remote Sensing Data with Wavelet Transform
• ExtremeC3Net: Extreme Lightweight Portrait Segmentation Networks using Advanced C3-modules
• Feature Pyramid Encoding Network for Real-time Semantic Segmentation
• Efficient Visual Recognition with Deep Neural Networks: A Survey on Recent Advances and New Directions
• C3: Concentrated-Comprehensive Convolution and its application to semantic segmentation
• Real-time CNN-based Segmentation Architecture for Ball Detection in a Single View Setup
• Segmenting Transparent Objects in the Wild
• MSCFNet: A Lightweight Network With Multi-Scale Context Fusion for Real-Time Semantic Segmentation
• DSNet: An Efficient CNN for Road Scene Segmentation
• CFPNet: Channel-wise Feature Pyramid for Real-Time Semantic Segmentation
• Predicting Terrain Mechanical Properties in Sight for Planetary Rovers with Semantic Clues
• FarSee-Net: Real-Time Semantic Segmentation by Efficient Multi-scale Context Aggregation and Feature Space Super-resolution
• Cross-Domain Transfer Learning with CoRTe: Consistent and Reliable Transfer from Black-Box to Lightweight Segmentation Model
• ShelfNet for Fast Semantic Segmentation
• Importance-Aware Semantic Segmentation with Efficient Pyramidal Context Network for Navigational Assistant Systems
• SINet: Extreme Lightweight Portrait Segmentation Networks with Spatial Squeeze Modules and Information Blocking Decoder
• RoadNet-RT: High Throughput CNN Architecture and SoC Design for Real-Time Road Segmentation
• Collaborative Global-Local Networks for Memory-Efficient Segmentation of Ultra-High Resolution Images
• Spatial Sampling Network for Fast Scene Understanding