Hierarchical Optimal Transport for Document Representation

Abstract

The ability to measure similarity between documents enables intelligent summarization and analysis of large corpora. Past distances between documents suffer from either an inability to incorporate semantic similarities between words or from scalability issues. As an alternative, we introduce hierarchical optimal transport as a meta-distance between documents, where documents are modeled as distributions over topics, which themselves are modeled as distributions over words. We then solve an optimal transport problem on the smaller topic space to compute a similarity score. We give conditions on the topics under which this construction defines a distance, and we relate it to the word mover's distance. We evaluate our technique for k-NN classification and show better interpretability and scalability with comparable performance to current methods at a fraction of the cost.

arXiv commentNeurIPS 2019

References in corpus (4)

• Scikit-learn: Machine Learning in Python
• Distributed Representations of Words and Phrases and their Compositionality
• Streaming Variational Bayes
• Distilled Wasserstein Learning for Word Embedding and Topic Modeling

Cited by in corpus (18)

• Geometric Dataset Distances via Optimal Transport
• Scalable Optimal Transport Methods in Machine Learning: A Contemporary Survey
• Learning Autoencoders with Relational Regularization
• Outlier-Robust Optimal Transport
• Re-evaluating Word Mover's Distance
• Scalable Nearest Neighbor Search for Optimal Transport
• Fast and Robust Comparison of Probability Measures in Heterogeneous Spaces
• Limit Distribution Theory for the Smooth 1-Wasserstein Distance with Applications
• Asymptotic Guarantees for Generative Modeling Based on the Smooth Wasserstein Distance
• Statistical Optimal Transport posed as Learning Kernel Embedding
• Gaussian-Smooth Optimal Transport: Metric Structure and Statistical Efficiency
• Neural Topic Model via Optimal Transport
• On Transportation of Mini-batches: A Hierarchical Approach
• Making transport more robust and interpretable by moving data through a small number of anchor points
• CO-Optimal Transport
• Order Constraints in Optimal Transport
• Hawkes Processes on Graphons
• Supervised Tree-Wasserstein Distance