Hyperspherical Prototype Networks

Abstract

This paper introduces hyperspherical prototype networks, which unify classification and regression with prototypes on hyperspherical output spaces. For classification, a common approach is to define prototypes as the mean output vector over training examples per class. Here, we propose to use hyperspheres as output spaces, with class prototypes defined a priori with large margin separation. We position prototypes through data-independent optimization, with an extension to incorporate priors from class semantics. By doing so, we do not require any prototype updating, we can handle any training size, and the output dimensionality is no longer constrained to the number of classes. Furthermore, we generalize to regression, by optimizing outputs as an interpolation between two prototypes on the hypersphere. Since both tasks are now defined by the same loss function, they can be jointly trained for multi-task problems. Experimentally, we show the benefit of hyperspherical prototype networks for classification, regression, and their combination over other prototype methods, softmax cross-entropy, and mean squared error approaches.

NeurIPS 2019

References in corpus (9)

• Distributed Representations of Words and Phrases and their Compositionality
• Prototypical Networks for Few-shot Learning
• SphereFace: Deep Hypersphere Embedding for Face Recognition
• Unsupervised Learning by Predicting Noise
• Poincaré GloVe: Hyperbolic Word Embeddings
• von Mises-Fisher Mixture Model-based Deep learning: Application to Face Verification
• OmniArt: Multi-task Deep Learning for Artistic Data Analysis
• Transferrable Prototypical Networks for Unsupervised Domain Adaptation
• Cosine Normalization: Using Cosine Similarity Instead of Dot Product in Neural Networks

Cited by in corpus (13)

• Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere
• Making Pre-trained Language Models Better Few-shot Learners
• TOAN: Target-Oriented Alignment Network for Fine-Grained Image Categorization with Few Labeled Samples
• OSLNet: Deep Small-Sample Classification with an Orthogonal Softmax Layer
• Leveraging Class Hierarchies with Metric-Guided Prototype Learning
• SecDD: Efficient and Secure Method for Remotely Training Neural Networks
• Hyperbolic Busemann Learning with Ideal Prototypes
• Regularizing Neural Networks via Minimizing Hyperspherical Energy
• Dissecting Supervised Contrastive Learning
• A Theory of Hyperbolic Prototype Learning
• Improvising the Learning of Neural Networks on Hyperspherical Manifold
• Attention Guided Cosine Margin For Overcoming Class-Imbalance in Few-Shot Road Object Detection
• Learning Representations on the Unit Sphere: Investigating Angular Gaussian and von Mises-Fisher Distributions for Online Continual Learning