Tackling the Local Bias in Federated Graph Learning

Abstract

Federated graph learning (FGL) has become an important research topic in response to the increasing scale and the distributed nature of graph-structured data in the real world. In FGL, a global graph is distributed across different clients, where each client holds a subgraph. Existing FGL methods often fail to effectively utilize cross-client edges, losing structural information during the training; additionally, local graphs often exhibit significant distribution divergence. These two issues make local models in FGL less desirable than in centralized graph learning, namely the local bias problem in this paper. To solve this problem, we propose a novel FGL framework to make the local models similar to the model trained in a centralized setting. Specifically, we design a distributed learning scheme, fully leveraging cross-client edges to aggregate information from other clients. In addition, we propose a label-guided sampling approach to alleviate the imbalanced local data and meanwhile, distinctly reduce the training overhead. Extensive experiments demonstrate that local bias can compromise the model performance and slow down the convergence during training. Experimental results also verify that our framework successfully mitigates local bias, achieving better performance than other baselines with lower time and memory overhead.

19 pages, 6 figures

References in corpus (12)

• Semi-Supervised Classification with Graph Convolutional Networks
• Inductive Representation Learning on Large Graphs
• Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks
• Open Graph Benchmark: Datasets for Machine Learning on Graphs
• FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling
• Federated Optimization in Heterogeneous Networks
• FedGraphNN: A Federated Learning System and Benchmark for Graph Neural Networks
• Layer-Dependent Importance Sampling for Training Deep and Large Graph Convolutional Networks
• GraphFL: A Federated Learning Framework for Semi-Supervised Node Classification on Graphs
• Distilling Self-Knowledge From Contrastive Links to Classify Graph Nodes Without Passing Messages
• Communication-Efficient Sampling for Distributed Training of Graph Convolutional Networks
• Distributed Training of Graph Convolutional Networks using Subgraph Approximation