Bayesian Neural Word Embedding

Abstract

Recently, several works in the domain of natural language processing presented successful methods for word embedding. Among them, the Skip-Gram with negative sampling, known also as word2vec, advanced the state-of-the-art of various linguistics tasks. In this paper, we propose a scalable Bayesian neural word embedding algorithm. The algorithm relies on a Variational Bayes solution for the Skip-Gram objective and a detailed step by step description is provided. We present experimental results that demonstrate the performance of the proposed algorithm for word analogy and similarity tasks on six different datasets and show it is competitive with the original Skip-Gram method.

Cited by in corpus (11)

• Survey of Computational Approaches to Lexical Semantic Change
• Correlation Coefficients and Semantic Textual Similarity
• Variational Bayesian Context-aware Representation for Grocery Recommendation
• Identification, Interpretability, and Bayesian Word Embeddings
• CB2CF: A Neural Multiview Content-to-Collaborative Filtering Model for Completely Cold Item Recommendations
• Augmenting and Tuning Knowledge Graph Embeddings
• Variational Bayesian Quantization
• Interpretable Word Embeddings via Informative Priors
• InverSynth: Deep Estimation of Synthesizer Parameter Configurations from Audio Signals
• Explainable Recommendations via Attentive Multi-Persona Collaborative Filtering
• A Quantum Field Theory of Representation Learning