BiSeNet V2: Bilateral Network with Guided Aggregation for Real-time Semantic Segmentation

Abstract

The low-level details and high-level semantics are both essential to the semantic segmentation task. However, to speed up the model inference, current approaches almost always sacrifice the low-level details, which leads to a considerable accuracy decrease. We propose to treat these spatial details and categorical semantics separately to achieve high accuracy and high efficiency for realtime semantic segmentation. To this end, we propose an efficient and effective architecture with a good trade-off between speed and accuracy, termed Bilateral Segmentation Network (BiSeNet V2). This architecture involves: (i) a Detail Branch, with wide channels and shallow layers to capture low-level details and generate high-resolution feature representation; (ii) a Semantic Branch, with narrow channels and deep layers to obtain high-level semantic context. The Semantic Branch is lightweight due to reducing the channel capacity and a fast-downsampling strategy. Furthermore, we design a Guided Aggregation Layer to enhance mutual connections and fuse both types of feature representation. Besides, a booster training strategy is designed to improve the segmentation performance without any extra inference cost. Extensive quantitative and qualitative evaluations demonstrate that the proposed architecture performs favourably against a few state-of-the-art real-time semantic segmentation approaches. Specifically, for a 2,048x1,024 input, we achieve 72.6% Mean IoU on the Cityscapes test set with a speed of 156 FPS on one NVIDIA GeForce GTX 1080 Ti card, which is significantly faster than existing methods, yet we achieve better segmentation accuracy.

16 pages, 10 figures, 9 tables. Code is available at https://git.io/BiSeNet

References in corpus (9)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Rethinking Atrous Convolution for Semantic Image Segmentation
• Conditional Random Fields as Recurrent Neural Networks
• ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation
• OCNet: Object Context Network for Scene Parsing
• Fast-SCNN: Fast Semantic Segmentation Network
• DABNet: Depth-wise Asymmetric Bottleneck for Real-time Semantic Segmentation

Cited by in corpus (17)

• Deep Dual-resolution Networks for Real-time and Accurate Semantic Segmentation of Road Scenes
• OFFSEG: A Semantic Segmentation Framework For Off-Road Driving
• Rethinking BiSeNet For Real-time Semantic Segmentation
• Lite-HRNet: A Lightweight High-Resolution Network
• OffRoadTranSeg: Semi-Supervised Segmentation using Transformers on OffRoad environments
• Rethinking Dilated Convolution for Real-time Semantic Segmentation
• A Survey on Deep Learning Methods for Semantic Image Segmentation in Real-Time
• AttaNet: Attention-Augmented Network for Fast and Accurate Scene Parsing
• Deep Relighting Networks for Image Light Source Manipulation
• Adversarial Multiscale Feature Learning for Overlapping Chromosome Segmentation
• Compact 3D Map-Based Monocular Localization Using Semantic Edge Alignment
• Trilateral Attention Network for Real-time Medical Image Segmentation
• Fast and Accurate Scene Parsing via Bi-direction Alignment Networks
• Retrieval and Localization with Observation Constraints
• RoadNet-RT: High Throughput CNN Architecture and SoC Design for Real-Time Road Segmentation
• Rethinking Lightweight Convolutional Neural Networks for Efficient and High-quality Pavement Crack Detection
• Reimagine BiSeNet for Real-Time Domain Adaptation in Semantic Segmentation