CoRGi: Content-Rich Graph Neural Networks with Attention

Abstract

Graph representations of a target domain often project it to a set of entities (nodes) and their relations (edges). However, such projections often miss important and rich information. For example, in graph representations used in missing value imputation, items - represented as nodes - may contain rich textual information. However, when processing graphs with graph neural networks (GNN), such information is either ignored or summarized into a single vector representation used to initialize the GNN. Towards addressing this, we present CoRGi, a GNN that considers the rich data within nodes in the context of their neighbors. This is achieved by endowing CoRGi's message passing with a personalized attention mechanism over the content of each node. This way, CoRGi assigns user-item-specific attention scores with respect to the words that appear in an item's content. We evaluate CoRGi on two edge-value prediction tasks and show that CoRGi is better at making edge-value predictions over existing methods, especially on sparse regions of the graph.

References in corpus (17)

• Distributed Representations of Words and Phrases and their Compositionality
• Auto-Encoding Variational Bayes
• Sequence to Sequence Learning with Neural Networks
• Semi-Supervised Classification with Graph Convolutional Networks
• Inductive Representation Learning on Large Graphs
• HuggingFace's Transformers: State-of-the-art Natural Language Processing
• A Structured Self-attentive Sentence Embedding
• A Deep Reinforced Model for Abstractive Summarization
• Graph Convolutional Matrix Completion
• Learning Convolutional Neural Networks for Graphs
• GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs
• Missing Data Imputation with Adversarially-trained Graph Convolutional Networks
• How to Find Your Friendly Neighborhood: Graph Attention Design with Self-Supervision
• Handling Missing Data with Graph Representation Learning
• Measuring and Improving the Use of Graph Information in Graph Neural Networks
• EDDI: Efficient Dynamic Discovery of High-Value Information with Partial VAE
• VAEM: a Deep Generative Model for Heterogeneous Mixed Type Data