Time-Dynamic Estimates of the Reliability of Deep Semantic Segmentation Networks

Abstract

In the semantic segmentation of street scenes with neural networks, the reliability of predictions is of highest interest. The assessment of neural networks by means of uncertainties is a common ansatz to prevent safety issues. As in applications like automated driving, video streams of images are available, we present a time-dynamic approach to investigating uncertainties and assessing the prediction quality of neural networks. We track segments over time and gather aggregated metrics per segment, thus obtaining time series of metrics from which we assess prediction quality. This is done by either classifying between intersection over union equal to 0 and greater than 0 or predicting the intersection over union directly. We study different models for these two tasks and analyze the influence of the time series length on the predictive power of our metrics.

References in corpus (3)

• Uncertainty and Interpretability in Convolutional Neural Networks for Semantic Segmentation of Colorectal Polyps
• Leveraging Uncertainty Estimates for Predicting Segmentation Quality
• Prediction Error Meta Classification in Semantic Segmentation: Detection via Aggregated Dispersion Measures of Softmax Probabilities

Cited by in corpus (9)

• A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges
• Run-Time Monitoring of Machine Learning for Robotic Perception: A Survey of Emerging Trends
• Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety
• Two Video Data Sets for Tracking and Retrieval of Out of Distribution Objects
• Uncertainty-weighted Loss Functions for Improved Adversarial Attacks on Semantic Segmentation
• Improving Video Instance Segmentation by Light-weight Temporal Uncertainty Estimates
• MetaFusion: Controlled False-Negative Reduction of Minority Classes in Semantic Segmentation
• False Negative Reduction in Video Instance Segmentation using Uncertainty Estimates
• Detecting Adversarial Attacks in Semantic Segmentation via Uncertainty Estimation: A Deep Analysis