Stochastic Neighbor Embedding with Gaussian and Student-t Distributions: Tutorial and Survey

Abstract

Stochastic Neighbor Embedding (SNE) is a manifold learning and dimensionality reduction method with a probabilistic approach. In SNE, every point is consider to be the neighbor of all other points with some probability and this probability is tried to be preserved in the embedding space. SNE considers Gaussian distribution for the probability in both the input and embedding spaces. However, t-SNE uses the Student-t and Gaussian distributions in these spaces, respectively. In this tutorial and survey paper, we explain SNE, symmetric SNE, t-SNE (or Cauchy-SNE), and t-SNE with general degrees of freedom. We also cover the out-of-sample extension and acceleration for these methods.

arXiv commentTo appear as a part of an upcoming academic book on dimensionality reduction and manifold learning. v2: applied readers' feedback

References in corpus (7)

• Efficient Estimation of Word Representations in Vector Space
• Distributed Representations of Words and Phrases and their Compositionality
• word2vec Explained: deriving Mikolov et al.'s negative-sampling word-embedding method
• word2vec Parameter Learning Explained
• Feature Selection and Feature Extraction in Pattern Analysis: A Literature Review
• Multidimensional Scaling, Sammon Mapping, and Isomap: Tutorial and Survey
• Stochastic Neighbor Embedding under f-divergences

Cited by in corpus (1)

• Factor Analysis, Probabilistic Principal Component Analysis, Variational Inference, and Variational Autoencoder: Tutorial and Survey