Focus Where It Matters: Graph Selective State Focused Attention Networks

Abstract

Traditional graph neural networks (GNNs) lack scalability and lose individual node characteristics due to over-smoothing, especially in the case of deeper networks. This results in sub-optimal feature representation, affecting the model's performance on tasks involving dynamically changing graphs. To address this issue, we present Graph Selective States Focused Attention Networks (GSANs) based neural network architecture for graph-structured data. The GSAN is enabled by multi-head masked self-attention (MHMSA) and selective state space modeling (S3M) layers to overcome the limitations of GNNs. In GSAN, the MHMSA allows GSAN to dynamically emphasize crucial node connections, particularly in evolving graph environments. The S3M layer enables the network to adjust dynamically in changing node states and improving predictions of node behavior in varying contexts without needing primary knowledge of the graph structure. Furthermore, the S3M layer enhances the generalization of unseen structures and interprets how node states influence link importance. With this, GSAN effectively outperforms inductive and transductive tasks and overcomes the issues that traditional GNNs experience. To analyze the performance behavior of GSAN, a set of state-of-the-art comparative experiments are conducted on graphs benchmark datasets, including $Cora$, $Citeseer$, $Pubmed$ network citation, and $protein-protein-interaction$ datasets, as an outcome, GSAN improved the classification accuracy by $1.56\%$, $8.94\%$, $0.37\%$, and $1.54\%$ on $F1-score$ respectively.

References in corpus (19)

• A Comprehensive Survey on Graph Neural Networks
• DeepWalk: Online Learning of Social Representations
• Simplifying Graph Convolutional Networks
• Representation Learning on Graphs with Jumping Knowledge Networks
• Predict then Propagate: Graph Neural Networks meet Personalized PageRank
• Foundations and modelling of dynamic networks using Dynamic Graph Neural Networks: A survey
• SIGN: Scalable Inception Graph Neural Networks
• Predicting citation counts based on deep neural network learning techniques
• Inductive Representation Learning in Temporal Networks via Causal Anonymous Walks
• Bi-Level Graph Structure Learning for Next POI Recommendation
• SPAGAN: Shortest Path Graph Attention Network
• STG-Mamba: Spatial-Temporal Graph Learning via Selective State Space Model
• A Comprehensive Survey of Dynamic Graph Neural Networks: Models, Frameworks, Benchmarks, Experiments and Challenges
• Adaptive Multi-layer Contrastive Graph Neural Networks
• Graph State-Space Models and Latent Relational Inference
• Hierarchical Attention Models for Multi-Relational Graphs
• Conformal Inductive Graph Neural Networks
• Graph Structure Learning with Bi-level Optimization
• Perturbation Ontology based Graph Attention Networks