Exponential Family Embeddings

Abstract

Word embeddings are a powerful approach for capturing semantic similarity among terms in a vocabulary. In this paper, we develop exponential family embeddings, a class of methods that extends the idea of word embeddings to other types of high-dimensional data. As examples, we studied neural data with real-valued observations, count data from a market basket analysis, and ratings data from a movie recommendation system. The main idea is to model each observation conditioned on a set of other observations. This set is called the context, and the way the context is defined is a modeling choice that depends on the problem. In language the context is the surrounding words; in neuroscience the context is close-by neurons; in market basket data the context is other items in the shopping cart. Each type of embedding model defines the context, the exponential family of conditional distributions, and how the latent embedding vectors are shared across data. We infer the embeddings with a scalable algorithm based on stochastic gradient descent. On all three applications - neural activity of zebrafish, users' shopping behavior, and movie ratings - we found exponential family embedding models to be more effective than other types of dimension reduction. They better reconstruct held-out data and find interesting qualitative structure.

Cited by in corpus (19)

• Deep Probabilistic Programming
• Improving Universal Sound Separation Using Sound Classification
• Learning Role-based Graph Embeddings
• Scalable bundling via dense product embeddings
• Nationality Classification Using Name Embeddings
• End-to-End Learning for the Deep Multivariate Probit Model
• DeepTriage: Automated Transfer Assistance for Incidents in Cloud Services
• Inferring Complementary Products from Baskets and Browsing Sessions
• Gaussian Word Embedding with a Wasserstein Distance Loss
• Medical Concept Embedding with Time-Aware Attention
• Context Aware Machine Learning
• Structured Embedding Models for Grouped Data
• A Dynamic Embedding Model of the Media Landscape
• Deep Multi-Species Embedding
• Query2Vec: An Evaluation of NLP Techniques for Generalized Workload Analytics
• Learning Rich Rankings
• Interpretable Word Embeddings via Informative Priors
• Representation Learning for Words and Entities
• Distributed Vector Representation Of Shopping Items, The Customer And Shopping Cart To Build A Three Fold Recommendation System