Comparative Analysis of Word Embeddings for Capturing Word Similarities

Abstract

Distributed language representation has become the most widely used technique for language representation in various natural language processing tasks. Most of the natural language processing models that are based on deep learning techniques use already pre-trained distributed word representations, commonly called word embeddings. Determining the most qualitative word embeddings is of crucial importance for such models. However, selecting the appropriate word embeddings is a perplexing task since the projected embedding space is not intuitive to humans. In this paper, we explore different approaches for creating distributed word representations. We perform an intrinsic evaluation of several state-of-the-art word embedding methods. Their performance on capturing word similarities is analysed with existing benchmark datasets for word pairs similarities. The research in this paper conducts a correlation analysis between ground truth word similarities and similarities obtained by different word embedding methods.

arXiv commentPart of the 6th International Conference on Natural Language Processing (NATP 2020)

References in corpus (7)

• Efficient Estimation of Word Representations in Vector Space
• Distributed Representations of Words and Phrases and their Compositionality
• Distributed Representations of Sentences and Documents
• A Survey on Recent Advances in Named Entity Recognition from Deep Learning models
• Two Step CCA: A new spectral method for estimating vector models of words
• An Ensemble Method to Produce High-Quality Word Embeddings (2016)
• A Comparison of Word Embeddings for the Biomedical Natural Language Processing

Cited by in corpus (2)

• Large Language Model Interface for Home Energy Management Systems
• A Comparative Analysis of Transformer and LSTM Models for Detecting Suicidal Ideation on Reddit