The Lovász-Softmax loss: A tractable surrogate for the optimization of the intersection-over-union measure in neural networks

Abstract

The Jaccard index, also referred to as the intersection-over-union score, is commonly employed in the evaluation of image segmentation results given its perceptual qualities, scale invariance - which lends appropriate relevance to small objects, and appropriate counting of false negatives, in comparison to per-pixel losses. We present a method for direct optimization of the mean intersection-over-union loss in neural networks, in the context of semantic image segmentation, based on the convex Lovász extension of submodular losses. The loss is shown to perform better with respect to the Jaccard index measure than the traditionally used cross-entropy loss. We show quantitative and qualitative differences between optimizing the Jaccard index per image versus optimizing the Jaccard index taken over an entire dataset. We evaluate the impact of our method in a semantic segmentation pipeline and show substantially improved intersection-over-union segmentation scores on the Pascal VOC and Cityscapes datasets using state-of-the-art deep learning segmentation architectures.

Accepted as a conference paper at CVPR 2018

References in corpus (2)

• Simultaneous Detection and Segmentation
• Proximal Backpropagation

Cited by in corpus (20)

• Multi-scale fully convolutional neural networks for histopathology image segmentation: from nuclear aberrations to the global tissue architecture
• A global method to identify trees outside of closed-canopy forests with medium-resolution satellite imagery
• Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentation
• (AF)2-S3Net: Attentive Feature Fusion with Adaptive Feature Selection for Sparse Semantic Segmentation Network
• Towards Semantic Segmentation of Urban-Scale 3D Point Clouds: A Dataset, Benchmarks and Challenges
• Automatic salt deposits segmentation: A deep learning approach
• ExtremeC3Net: Extreme Lightweight Portrait Segmentation Networks using Advanced C3-modules
• DRINet++: Efficient Voxel-as-point Point Cloud Segmentation
• LiDAR-based Panoptic Segmentation via Dynamic Shifting Network
• Panoptic-PolarNet: Proposal-free LiDAR Point Cloud Panoptic Segmentation
• Semi-Supervised Segmentation of Salt Bodies in Seismic Images using an Ensemble of Convolutional Neural Networks
• The 1st Agriculture-Vision Challenge: Methods and Results
• Learning of Image Dehazing Models for Segmentation Tasks
• AutoLoss-Zero: Searching Loss Functions from Scratch for Generic Tasks
• Quality-Aware Network for Human Parsing
• IoU is not submodular
• Reducing the feature divergence of RGB and near-infrared images using Switchable Normalization
• Multi Receptive Field Network for Semantic Segmentation
• Automated risk classification of colon biopsies based on semantic segmentation of histopathology images
• LiDAR-based Recurrent 3D Semantic Segmentation with Temporal Memory Alignment