Exploring Uncertainty in Conditional Multi-Modal Retrieval Systems
arXiv:1901.07702
Abstract
We cast visual retrieval as a regression problem by posing triplet loss as a regression loss. This enables epistemic uncertainty estimation using dropout as a Bayesian approximation framework in retrieval. Accordingly, Monte Carlo (MC) sampling is leveraged to boost retrieval performance. Our approach is evaluated on two applications: person re-identification and autonomous car driving. Comparable state-of-the-art results are achieved on multiple datasets for the former application. We leverage the Honda driving dataset (HDD) for autonomous car driving application. It provides multiple modalities and similarity notions for ego-motion action understanding. Hence, we present a multi-modal conditional retrieval network. It disentangles embeddings into separate representations to encode different similarities. This form of joint learning eliminates the need to train multiple independent networks without any performance degradation. Quantitative evaluation highlights our approach competence, achieving 6% improvement in a highly uncertain environment.
References in corpus (17)
- Adam: A Method for Stochastic Optimization
- Two-Stream Convolutional Networks for Action Recognition in Videos
- What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
- In Defense of the Triplet Loss for Person Re-Identification
- Person Re-identification: Past, Present and Future
- Bayesian Convolutional Neural Networks with Bernoulli Approximate Variational Inference
- Pedestrian Alignment Network for Large-scale Person Re-identification
- Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics
- Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding
- Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net
- One Model To Learn Them All
- Looking Beyond Appearances: Synthetic Training Data for Deep CNNs in Re-identification
- Unlabeled Samples Generated by GAN Improve the Person Re-identification Baseline in vitro
- Temporal Context Network for Activity Localization in Videos
- Dropout as a Bayesian Approximation: Appendix
- Two Stream Self-Supervised Learning for Action Recognition
- Efficient Uncertainty Estimation for Semantic Segmentation in Videos