Dynamic Embedding on Textual Networks via a Gaussian Process

Abstract

Textual network embedding aims to learn low-dimensional representations of text-annotated nodes in a graph. Prior work in this area has typically focused on fixed graph structures; however, real-world networks are often dynamic. We address this challenge with a novel end-to-end node-embedding model, called Dynamic Embedding for Textual Networks with a Gaussian Process (DetGP). After training, DetGP can be applied efficiently to dynamic graphs without re-training or backpropagation. The learned representation of each node is a combination of textual and structural embeddings. Because the structure is allowed to be dynamic, our method uses the Gaussian process to take advantage of its non-parametric properties. To use both local and global graph structures, diffusion is used to model multiple hops between neighbors. The relative importance of global versus local structure for the embeddings is learned automatically. With the non-parametric nature of the Gaussian process, updating the embeddings for a changed graph structure requires only a forward pass through the learned model. Considering link prediction and node classification, experiments demonstrate the empirical effectiveness of our method compared to baseline approaches. We further show that DetGP can be straightforwardly and efficiently applied to dynamic textual networks.

Accepted for presentation at the Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI-20)

References in corpus (15)

• Inductive Representation Learning on Large Graphs
• Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation
• Universal Sentence Encoder
• Variational Graph Auto-Encoders
• Predicting multicellular function through multi-layer tissue networks
• Attributed Network Embedding for Learning in a Dynamic Environment
• DynGEM: Deep Embedding Method for Dynamic Graphs
• Watch Your Step: Learning Node Embeddings via Graph Attention
• Towards Universal Paraphrastic Sentence Embeddings
• A General Framework for Content-enhanced Network Representation Learning
• Graph Neural Networks for Social Recommendation
• Representation Learning over Dynamic Graphs
• Diffusion Maps for Textual Network Embedding
• SPINE: Structural Identity Preserved Inductive Network Embedding
• NSCaching: Simple and Efficient Negative Sampling for Knowledge Graph Embedding