A Hybrid Similarity-Aware Graph Neural Network with Transformer for Node Classification

Abstract

Node classification has gained significant importance in graph deep learning with real-world applications such as recommendation systems, drug discovery, and citation networks. Graph Convolutional Networks and Graph Transformers have achieved superior performance in node classification tasks. However, the key concern with Graph Convolutional Networks is over-squashing, which limits their ability to capture long-range dependencies in the network. Additionally, Graph Transformers face scalability challenges, making it difficult to process large graphs efficiently. To address this, we propose a novel framework, A Hybrid SImilarity-Aware Graph Neural Network with Transformer for Node Classification (SIGNNet), which capitalizes on local and global structural information, enhances the model's capability to effectively capture fine-grained relationships and broader contextual patterns within the graph structure. The proposed method leverages Graph Convolutional Networks alongside a score-based mechanism to effectively capture local and global node interactions while addressing the limitations of over-squashing. Our proposed method employs a novel Personalized PageRank-based node sampling method to address scalability issues by generating subgraphs of nodes. Additionally, SIGNNet incorporates a novel attention mechanism, Structure-Aware Multi-Head Attention (SA-MHA), which integrates node structural information for informed attention weighting, enabling the model to prioritize nodes based on topological significance. Extensive experiments demonstrate the significant improvements achieved by the proposed method over existing state-of-the-art methods, with average accuracy gains of 6.03%, 5.47%, 4.78%, 19.10%, 19.61%, 7.22%, 19.54%, and 14.94% on Cora, Citeseer, CS, Wisconsin, Texas, Actor, Cornell and Chameleon datasets, respectively.

References in corpus (8)

• Semi-Supervised Classification with Graph Convolutional Networks
• Fast Graph Representation Learning with PyTorch Geometric
• Efficient Algorithms for Personalized PageRank Computation: A Survey
• Is Heterophily A Real Nightmare For Graph Neural Networks To Do Node Classification?
• UniG-Encoder: A Universal Feature Encoder for Graph and Hypergraph Node Classification
• A Context-aware Attention and Graph Neural Network-based Multimodal Framework for Misogyny Detection
• Unleashing the Power of Transformer for Graphs
• Tokenized Graph Transformer with Neighborhood Augmentation for Node Classification in Large Graphs

Cited by in corpus (3)

• T-MPEDNet: Unveiling the Synergy of Transformer-aware Multiscale Progressive Encoder-Decoder Network with Feature Recalibration for Tumor and Liver Segmentation
• An Explainable Deep Neural Network with Frequency-Aware Channel and Spatial Refinement for Flood Prediction in Sustainable Cities
• Emotion-aware Dual Cross-Attentive Neural Network with Label Fusion for Stance Detection in Misinformative Social Media Content