Knowledge Adaptation for Efficient Semantic Segmentation

Abstract

Both accuracy and efficiency are of significant importance to the task of semantic segmentation. Existing deep FCNs suffer from heavy computations due to a series of high-resolution feature maps for preserving the detailed knowledge in dense estimation. Although reducing the feature map resolution (i.e., applying a large overall stride) via subsampling operations (e.g., pooling and convolution striding) can instantly increase the efficiency, it dramatically decreases the estimation accuracy. To tackle this dilemma, we propose a knowledge distillation method tailored for semantic segmentation to improve the performance of the compact FCNs with large overall stride. To handle the inconsistency between the features of the student and teacher network, we optimize the feature similarity in a transferred latent domain formulated by utilizing a pre-trained autoencoder. Moreover, an affinity distillation module is proposed to capture the long-range dependency by calculating the non-local interactions across the whole image. To validate the effectiveness of our proposed method, extensive experiments have been conducted on three popular benchmarks: Pascal VOC, Cityscapes and Pascal Context. Built upon a highly competitive baseline, our proposed method can improve the performance of a student network by 2.5\% (mIOU boosts from 70.2 to 72.7 on the cityscapes test set) and can train a better compact model with only 8\% float operations (FLOPS) of a model that achieves comparable performances.

Accepted to IEEE Conf. Computer Vision and Pattern Recognition, 2019

References in corpus (20)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Distilling the Knowledge in a Neural Network
• Rethinking Atrous Convolution for Semantic Image Segmentation
• Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
• Deep Residual Learning for Image Recognition
• Conditional Random Fields as Recurrent Neural Networks
• FitNets: Hints for Thin Deep Nets
• Multi-Scale Context Aggregation by Dilated Convolutions
• ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation
• The Cityscapes Dataset for Semantic Urban Scene Understanding
• Fully Convolutional Networks for Semantic Segmentation
• Like What You Like: Knowledge Distill via Neuron Selectivity Transfer
• Pyramid Scene Parsing Network
• Wider or Deeper: Revisiting the ResNet Model for Visual Recognition
• Semantic Image Segmentation via Deep Parsing Network
• Non-local Neural Networks
• ICNet for Real-Time Semantic Segmentation on High-Resolution Images
• Paraphrasing Complex Network: Network Compression via Factor Transfer
• Deep Learning of Nonnegativity-Constrained Autoencoders for Enhanced Understanding of Data
• ExFuse: Enhancing Feature Fusion for Semantic Segmentation

Cited by in corpus (6)

• Knowledge Distillation: A Survey
• COVID-19 Chest CT Image Segmentation -- A Deep Convolutional Neural Network Solution
• Switchable Precision Neural Networks
• Temporally Distributed Networks for Fast Video Semantic Segmentation
• Effective Training of Convolutional Neural Networks with Low-bitwidth Weights and Activations
• Decoders Matter for Semantic Segmentation: Data-Dependent Decoding Enables Flexible Feature Aggregation