Stochastic Prototype Embeddings
arXiv:1909.11702
Abstract
Supervised deep-embedding methods project inputs of a domain to a representational space in which same-class instances lie near one another and different-class instances lie far apart. We propose a probabilistic method that treats embeddings as random variables. Extending a state-of-the-art deterministic method, Prototypical Networks (Snell et al., 2017), our approach supposes the existence of a class prototype around which class instances are Gaussian distributed. The prototype posterior is a product distribution over labeled instances, and query instances are classified by marginalizing relative prototype proximity over embedding uncertainty. We describe an efficient sampler for approximate inference that allows us to train the model at roughly the same space and time cost as its deterministic sibling. Incorporating uncertainty improves performance on few-shot learning and gracefully handles label noise and out-of-distribution inputs. Compared to the state-of-the-art stochastic method, Hedged Instance Embeddings (Oh et al., 2019), we achieve superior large- and open-set classification accuracy. Our method also aligns class-discriminating features with the axes of the embedding space, yielding an interpretable, disentangled representation.
15 pages, 8 figures
References in corpus (9)
- Auto-Encoding Variational Bayes
- Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
- Prototypical Networks for Few-shot Learning
- A Simple Neural Attentive Meta-Learner
- Learning Deep Embeddings with Histogram Loss
- Targeting Ultimate Accuracy: Face Recognition via Deep Embedding
- Gaussian Prototypical Networks for Few-Shot Learning on Omniglot
- Bayesian representation learning with oracle constraints
- Adapted Deep Embeddings: A Synthesis of Methods for -Shot Inductive Transfer Learning