Synthesize then Compare: Detecting Failures and Anomalies for Semantic Segmentation
arXiv:2003.08440
Abstract
The ability to detect failures and anomalies are fundamental requirements for building reliable systems for computer vision applications, especially safety-critical applications of semantic segmentation, such as autonomous driving and medical image analysis. In this paper, we systematically study failure and anomaly detection for semantic segmentation and propose a unified framework, consisting of two modules, to address these two related problems. The first module is an image synthesis module, which generates a synthesized image from a segmentation layout map, and the second is a comparison module, which computes the difference between the synthesized image and the input image. We validate our framework on three challenging datasets and improve the state-of-the-arts by large margins, \emph{i.e.}, 6% AUPR-Error on Cityscapes, 7% Pearson correlation on pancreatic tumor segmentation in MSD and 20% AUPR on StreetHazards anomaly segmentation.
arXiv commentECCV 2020 Oral. The first two authors contributed equally to this work. Code available at https://github.com/YingdaXia/SynthCP
References in corpus (14)
- Auto-Encoding Variational Bayes
- Conditional Generative Adversarial Nets
- What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
- A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
- A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
- A large annotated medical image dataset for the development and evaluation of segmentation algorithms
- Deep Autoencoding Models for Unsupervised Anomaly Segmentation in Brain MR Images
- Deep Anomaly Detection with Outlier Exposure
- Learning Confidence for Out-of-Distribution Detection in Neural Networks
- To Trust Or Not To Trust A Classifier
- Learning to Predict Layout-to-image Conditional Convolutions for Semantic Image Synthesis
- Discriminative out-of-distribution detection for semantic segmentation
- Uncertainty-driven Sanity Check: Application to Postoperative Brain Tumor Cavity Segmentation
- Robust Semantic Segmentation with Ladder-DenseNet Models