Word Representations via Gaussian Embedding

Abstract

Current work in lexical distributed representations maps each word to a point vector in low-dimensional space. Mapping instead to a density provides many interesting advantages, including better capturing uncertainty about a representation and its relationships, expressing asymmetries more naturally than dot product or cosine similarity, and enabling more expressive parameterization of decision boundaries. This paper advocates for density-based distributed embeddings and presents a method for learning representations in the space of Gaussian distributions. We compare performance on various word embedding benchmarks, investigate the ability of these embeddings to model entailment and other asymmetric relationships, and explore novel properties of the representation.

12 pages, published as conference paper at ICLR 2015

Cited by in corpus (52)

• Deep Gaussian Embedding of Graphs: Unsupervised Inductive Learning via Ranking
• Variational Federated Multi-Task Learning
• A Dual Embedding Space Model for Document Ranking
• Order-Embeddings of Images and Language
• Beta Embeddings for Multi-Hop Logical Reasoning in Knowledge Graphs
• Modeling Uncertainty with Hedged Instance Embedding
• Generative Models of Visually Grounded Imagination
• Self-Supervised Hyperboloid Representations from Logical Queries over Knowledge Graphs
• Generalizing Point Embeddings using the Wasserstein Space of Elliptical Distributions
• Quantum Mathematics in Artificial Intelligence
• Faithful Embeddings for Knowledge Base Queries
• TaxoCom: Topic Taxonomy Completion with Hierarchical Discovery of Novel Topic Clusters
• Gaussian Attention Model and Its Application to Knowledge Base Embedding and Question Answering
• Multi-Modal Bayesian Embeddings for Learning Social Knowledge Graphs
• View-Invariant, Occlusion-Robust Probabilistic Embedding for Human Pose
• Cognitive Database: A Step towards Endowing Relational Databases with Artificial Intelligence Capabilities
• Multimodal Word Distributions
• Deep Learning for Learning Graph Representations
• Entity Embeddings with Conceptual Subspaces as a Basis for Plausible Reasoning
• Towards logical negation for compositional distributional semantics
• Gaussian Word Embedding with a Wasserstein Distance Loss
• Neural Graph Embedding Methods for Natural Language Processing
• Enabling Cognitive Intelligence Queries in Relational Databases using Low-dimensional Word Embeddings
• Gaussian Mixture Embeddings for Multiple Word Prototypes
• Fast and Reliable Probabilistic Face Embeddings in the Wild
• Personalised Federated Learning On Heterogeneous Feature Spaces
• Bootstrap Equilibrium and Probabilistic Speaker Representation Learning for Self-supervised Speaker Verification
• Modeling Sequences as Distributions with Uncertainty for Sequential Recommendation
• Probabilistic Embeddings for Cross-Modal Retrieval
• View-Invariant Probabilistic Embedding for Human Pose
• An Inductive Bias for Distances: Neural Nets that Respect the Triangle Inequality
• Hierarchical Density Order Embeddings
• Mixture-of-tastes Models for Representing Users with Diverse Interests
• WordRank: Learning Word Embeddings via Robust Ranking
• NEAT: A Label Noise-resistant Complementary Item Recommender System with Trustworthy Evaluation
• Neural Variational Inference For Estimating Uncertainty in Knowledge Graph Embeddings
• Density Matching for Bilingual Word Embedding
• Modeling Fine-Grained Entity Types with Box Embeddings
• Zero-Shot Clinical Acronym Expansion via Latent Meaning Cells
• Reducing Semantic Ambiguity In Domain Adaptive Semantic Segmentation Via Probabilistic Prototypical Pixel Contrast
• Term Definitions Help Hypernymy Detection
• Empirical Study of Named Entity Recognition Performance Using Distribution-aware Word Embedding
• English Out-of-Vocabulary Lexical Evaluation Task
• Contextual Salience for Fast and Accurate Sentence Vectors
• Interpretable Word Embeddings via Informative Priors
• Learning Graph-Based Priors for Generalized Zero-Shot Learning
• Out-of-Category Document Identification Using Target-Category Names as Weak Supervision
• Learning Probabilistic Sentence Representations from Paraphrases
• Exploration on Grounded Word Embedding: Matching Words and Images with Image-Enhanced Skip-Gram Model
• Convolutional Gaussian Embeddings for Personalized Recommendation with Uncertainty
• Learning Multi-Sense Word Distributions using Approximate Kullback-Leibler Divergence
• Bayesian Metric Learning for Uncertainty Quantification in Image Retrieval