FedGraphNN: A Federated Learning System and Benchmark for Graph Neural Networks

Abstract

Graph Neural Network (GNN) research is rapidly growing thanks to the capacity of GNNs in learning distributed representations from graph-structured data. However, centralizing a massive amount of real-world graph data for GNN training is prohibitive due to privacy concerns, regulation restrictions, and commercial competitions. Federated learning (FL), a trending distributed learning paradigm, provides possibilities to solve this challenge while preserving data privacy. Despite recent advances in vision and language domains, there is no suitable platform for the FL of GNNs. To this end, we introduce FedGraphNN, an open FL benchmark system that can facilitate research on federated GNNs. FedGraphNN is built on a unified formulation of graph FL and contains a wide range of datasets from different domains, popular GNN models, and FL algorithms, with secure and efficient system support. Particularly for the datasets, we collect, preprocess, and partition 36 datasets from 7 domains, including both publicly available ones and specifically obtained ones such as hERG and Tencent. Our empirical analysis showcases the utility of our benchmark system, while exposing significant challenges in graph FL: federated GNNs perform worse in most datasets with a non-IID split than centralized GNNs; the GNN model that attains the best result in the centralized setting may not maintain its advantage in the FL setting. These results imply that more research efforts are needed to unravel the mystery behind federated GNNs. Moreover, our system performance analysis demonstrates that the FedGraphNN system is computationally efficient and secure to large-scale graphs datasets. We maintain the source code at https://github.com/FedML-AI/FedGraphNN.

Our shorter versions are accepted to ICLR 2021 Workshop on Distributed and Private Machine Learning(DPML) and MLSys 2021 GNNSys Workshop on Graph Neural Networks and Systems. The full version is under review

References in corpus (23)

• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Inductive Representation Learning on Large Graphs
• Neural Message Passing for Quantum Chemistry
• Simplifying Graph Convolutional Networks
• Federated Learning for Mobile Keyboard Prediction
• Pitfalls of Graph Neural Network Evaluation
• FedML: A Research Library and Benchmark for Federated Machine Learning
• A generic framework for privacy preserving deep learning
• LEAF: A Benchmark for Federated Settings
• Adaptive Federated Optimization
• SocialGCN: An Efficient Graph Convolutional Network based Model for Social Recommendation
• Subgraph Federated Learning with Missing Neighbor Generation
• Central Server Free Federated Learning over Single-sided Trust Social Networks
• DIG: A Turnkey Library for Diving into Graph Deep Learning Research
• FedNER: Privacy-preserving Medical Named Entity Recognition with Federated Learning
• FedCV: A Federated Learning Framework for Diverse Computer Vision Tasks
• GraphFL: A Federated Learning Framework for Semi-Supervised Node Classification on Graphs
• Towards Non-I.I.D. and Invisible Data with FedNAS: Federated Deep Learning via Neural Architecture Search
• SpreadGNN: Serverless Multi-task Federated Learning for Graph Neural Networks
• Federated Dynamic GNN with Secure Aggregation
• DistDGL: Distributed Graph Neural Network Training for Billion-Scale Graphs
• Secure Aggregation with Heterogeneous Quantization in Federated Learning
• Cascade-BGNN: Toward Efficient Self-supervised Representation Learning on Large-scale Bipartite Graphs

Cited by in corpus (17)

• DGRec: Graph Neural Network for Recommendation with Diversified Embedding Generation
• FedNI: Federated Graph Learning with Network Inpainting for Population-Based Disease Prediction
• Trustworthy Graph Neural Networks: Aspects, Methods and Trends
• Subgraph Federated Learning with Missing Neighbor Generation
• Artificial Intelligence of Things: A Survey
• Federated Graph Classification over Non-IID Graphs
• FedCV: A Federated Learning Framework for Diverse Computer Vision Tasks
• Graph Federated Learning for CIoT Devices in Smart Home Applications
• Graph Mining for Cybersecurity: A Survey
• SpreadGNN: Serverless Multi-task Federated Learning for Graph Neural Networks
• Federated Graph Learning -- A Position Paper
• STFL: A Temporal-Spatial Federated Learning Framework for Graph Neural Networks
• Independent Distribution Regularization for Private Graph Embedding
• Towards Representation Identical Privacy-Preserving Graph Neural Network via Split Learning
• 2SFGL: A Simple And Robust Protocol For Graph-Based Fraud Detection
• FedGraph: Federated Graph Learning with Intelligent Sampling
• Tackling the Local Bias in Federated Graph Learning