Attention Models in Graphs: A Survey

Abstract

Graph-structured data arise naturally in many different application domains. By representing data as graphs, we can capture entities (i.e., nodes) as well as their relationships (i.e., edges) with each other. Many useful insights can be derived from graph-structured data as demonstrated by an ever-growing body of work focused on graph mining. However, in the real-world, graphs can be both large - with many complex patterns - and noisy which can pose a problem for effective graph mining. An effective way to deal with this issue is to incorporate "attention" into graph mining solutions. An attention mechanism allows a method to focus on task-relevant parts of the graph, helping it to make better decisions. In this work, we conduct a comprehensive and focused survey of the literature on the emerging field of graph attention models. We introduce three intuitive taxonomies to group existing work. These are based on problem setting (type of input and output), the type of attention mechanism used, and the task (e.g., graph classification, link prediction, etc.). We motivate our taxonomies through detailed examples and use each to survey competing approaches from a unique standpoint. Finally, we highlight several challenges in the area and discuss promising directions for future work.

Cited by in corpus (13)

• Simplifying Graph Convolutional Networks
• A General Survey on Attention Mechanisms in Deep Learning
• Attention in Natural Language Processing
• Learning Combinatorial Optimization on Graphs: A Survey with Applications to Networking
• Relational Graph Attention Networks
• Recommender systems based on graph embedding techniques: A comprehensive review
• Wind Park Power Prediction: Attention-Based Graph Networks and Deep Learning to Capture Wake Losses
• Global Attention Improves Graph Networks Generalization
• The Impact of Global Structural Information in Graph Neural Networks Applications
• Identifying extra high frequency gravitational waves generated from oscillons with cuspy potentials using deep neural networks
• PushNet: Efficient and Adaptive Neural Message Passing
• HiCoRe: Visual Hierarchical Context-Reasoning
• geoGAT: Graph Model Based on Attention Mechanism for Geographic Text Classification