Learning in an Uncertain World: Representing Ambiguity Through Multiple Hypotheses

Abstract

Many prediction tasks contain uncertainty. In some cases, uncertainty is inherent in the task itself. In future prediction, for example, many distinct outcomes are equally valid. In other cases, uncertainty arises from the way data is labeled. For example, in object detection, many objects of interest often go unlabeled, and in human pose estimation, occluded joints are often labeled with ambiguous values. In this work we focus on a principled approach for handling such scenarios. In particular, we propose a framework for reformulating existing single-prediction models as multiple hypothesis prediction (MHP) models and an associated meta loss and optimization procedure to train them. To demonstrate our approach, we consider four diverse applications: human pose estimation, future prediction, image classification and segmentation. We find that MHP models outperform their single-hypothesis counterparts in all cases, and that MHP models simultaneously expose valuable insights into the variability of predictions.

arXiv commentICCV 2017

References in corpus (10)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Unsupervised Learning of Video Representations using LSTMs
• Fully Convolutional Networks for Semantic Segmentation
• CNN: Single-label to Multi-label
• Deep Label Distribution Learning with Label Ambiguity
• NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis
• Deep Convolutional Ranking for Multilabel Image Annotation
• CNN-RNN: A Unified Framework for Multi-label Image Classification
• Exploit Bounding Box Annotations for Multi-label Object Recognition

Cited by in corpus (14)

• Overcoming Limitations of Mixture Density Networks: A Sampling and Fitting Framework for Multimodal Future Prediction
• Stochastic Segmentation Networks: Modelling Spatially Correlated Aleatoric Uncertainty
• Injecting Knowledge in Data-driven Vehicle Trajectory Predictors
• Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networks
• Theoretical analysis and experimental validation of volume bias of soft Dice optimized segmentation maps in the context of inherent uncertainty
• Ensembling Low Precision Models for Binary Biomedical Image Segmentation
• X-Ray: Mechanical Search for an Occluded Object by Minimizing Support of Learned Occupancy Distributions
• FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Cameras
• Pathological Retinal Region Segmentation From OCT Images Using Geometric Relation Based Augmentation
• Improving Regression Performance with Distributional Losses
• Deep Context-Aware Novelty Detection
• CT Image Synthesis Using Weakly Supervised Segmentation and Geometric Inter-Label Relations For COVID Image Analysis
• Inferring Distributions Over Depth from a Single Image
• Visual Attention in Imaginative Agents