Modeling Uncertainty with Hedged Instance Embedding

Abstract

Instance embeddings are an efficient and versatile image representation that facilitates applications like recognition, verification, retrieval, and clustering. Many metric learning methods represent the input as a single point in the embedding space. Often the distance between points is used as a proxy for match confidence. However, this can fail to represent uncertainty arising when the input is ambiguous, e.g., due to occlusion or blurriness. This work addresses this issue and explicitly models the uncertainty by hedging the location of each input in the embedding space. We introduce the hedged instance embedding (HIB) in which embeddings are modeled as random variables and the model is trained under the variational information bottleneck principle. Empirical results on our new N-digit MNIST dataset show that our method leads to the desired behavior of hedging its bets across the embedding space upon encountering ambiguous inputs. This results in improved performance for image matching and classification tasks, more structure in the learned embedding space, and an ability to compute a per-exemplar uncertainty measure that is correlated with downstream performance.

arXiv comment15 pages, 11 figures, updated version of ICLR'19

References in corpus (11)

• Auto-Encoding Variational Bayes
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
• The information bottleneck method
• Word Representations via Gaussian Embedding
• Uncertainty in the Variational Information Bottleneck
• No Fuss Distance Metric Learning using Proxies
• Bayesian representation learning with oracle constraints
• Rethinking Feature Distribution for Loss Functions in Image Classification
• Fully-Convolutional Siamese Networks for Object Tracking
• Gradient-Leaks: Understanding and Controlling Deanonymization in Federated Learning

Cited by in corpus (13)

• A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges
• Polysemous Visual-Semantic Embedding for Cross-Modal Retrieval
• View-Invariant, Occlusion-Robust Probabilistic Embedding for Human Pose
• Temperature as Uncertainty in Contrastive Learning
• Improving Back-Translation with Uncertainty-based Confidence Estimation
• Fast and Reliable Probabilistic Face Embeddings in the Wild
• Probabilistic Embeddings for Cross-Modal Retrieval
• Bootstrap Equilibrium and Probabilistic Speaker Representation Learning for Self-supervised Speaker Verification
• View-Invariant Probabilistic Embedding for Human Pose
• Automatic Differentiation Variational Inference with Mixtures
• Reducing Semantic Ambiguity In Domain Adaptive Semantic Segmentation Via Probabilistic Prototypical Pixel Contrast
• Bayesian Metric Learning for Uncertainty Quantification in Image Retrieval
• Multi-view Alignment and Generation in CCA via Consistent Latent Encoding