word2vec Skip-Gram with Negative Sampling is a Weighted Logistic PCA

Abstract

We show that the skip-gram formulation of word2vec trained with negative sampling is equivalent to a weighted logistic PCA. This connection allows us to better understand the objective, compare it to other word embedding methods, and extend it to higher dimensional models.

References in corpus (1)

• word2vec Explained: deriving Mikolov et al.'s negative-sampling word-embedding method

Cited by in corpus (1)

• Revisiting Skip-Gram Negative Sampling Model with Rectification