Handling Feature Heterogeneity with Learnable Graph Patches

Abstract

In recent years, the rapid development of foundation models and graph pre-training technologies has spurred increasing interest in constructing a universal pre-trained graph model or Graph Foundation Model (GFM). However, a significant challenge is that existing models are unable to address feature heterogeneity in graph data without textual information, which hinders the transferability of graph models across different datasets. To bridge this gap, we propose the concept of learnable graph patches, which we regard as the smallest semantic units of any graph data. We decompose the graph into learnable graph patches by unfolding the node features and constructing corresponding patch structures separately. We then design a framework that mines transferable information from graph data across domains. Specifically, after extracting graph patches, we propose a patch encoder to extract knowledge from each unit and a patch aggregator to learn how the units are combined into a whole. Due to its domain-agnostic nature, the model can be applied to downstream data across different domains. Furthermore, we analyze the connection between our method and existing graph models, as well as the transferability of the node embeddings it generates. Empirically, our method not only achieves the capability to use multi-domain graphs for pre-training, but also shows enhanced performance across various downstream datasets and tasks. Moreover, we observe consistent improvement in downstream performance as the volume of pre-training data increases.

Accepted at KDD 2025

References in corpus (7)

• Molecular Contrastive Learning of Representations via Graph Neural Networks
• GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training
• SimGRACE: A Simple Framework for Graph Contrastive Learning without Data Augmentation
• Recipe for a General, Powerful, Scalable Graph Transformer
• Adversarial Graph Augmentation to Improve Graph Contrastive Learning
• Transfer Learning of Graph Neural Networks with Ego-graph Information Maximization
• SLAPS: Self-Supervision Improves Structure Learning for Graph Neural Networks