Stochastic Multiple Choice Learning for Training Diverse Deep Ensembles

Abstract

Many practical perception systems exist within larger processes that include interactions with users or additional components capable of evaluating the quality of predicted solutions. In these contexts, it is beneficial to provide these oracle mechanisms with multiple highly likely hypotheses rather than a single prediction. In this work, we pose the task of producing multiple outputs as a learning problem over an ensemble of deep networks -- introducing a novel stochastic gradient descent based approach to minimize the loss with respect to an oracle. Our method is simple to implement, agnostic to both architecture and loss function, and parameter-free. Our approach achieves lower oracle error compared to existing methods on a wide range of tasks and deep architectures. We also show qualitatively that the diverse solutions produced often provide interpretable representations of task ambiguity.

References in corpus (2)

• Distilling the Knowledge in a Neural Network
• Network of Experts for Large-Scale Image Categorization

Cited by in corpus (8)

• Transformation-Grounded Image Generation Network for Novel 3D View Synthesis
• Message Passing Multi-Agent GANs
• Uncertainty Estimates and Multi-Hypotheses Networks for Optical Flow
• Generalized Negative Correlation Learning for Deep Ensembling
• Diversity and Generalization in Neural Network Ensembles
• Towards Diverse Perspective Learning with Selection over Multiple Temporal Poolings
• Amended Cross Entropy Cost: Framework For Explicit Diversity Encouragement
• DiverseNet: When One Right Answer is not Enough