FedGAN: Federated Generative Adversarial Networks for Distributed Data

Abstract

We propose Federated Generative Adversarial Network (FedGAN) for training a GAN across distributed sources of non-independent-and-identically-distributed data sources subject to communication and privacy constraints. Our algorithm uses local generators and discriminators which are periodically synced via an intermediary that averages and broadcasts the generator and discriminator parameters. We theoretically prove the convergence of FedGAN with both equal and two time-scale updates of generator and discriminator, under standard assumptions, using stochastic approximations and communication efficient stochastic gradient descents. We experiment FedGAN on toy examples (2D system, mixed Gaussian, and Swiss role), image datasets (MNIST, CIFAR-10, and CelebA), and time series datasets (household electricity consumption and electric vehicle charging sessions). We show FedGAN converges and has similar performance to general distributed GAN, while reduces communication complexity. We also show its robustness to reduced communications.

23 pages, 10 figures

References in corpus (18)

• Conditional Generative Adversarial Nets
• Communication-Efficient Learning of Deep Networks from Decentralized Data
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Federated Learning: Strategies for Improving Communication Efficiency
• Conditional Image Synthesis With Auxiliary Classifier GANs
• On the Convergence of FedAvg on Non-IID Data
• Unrolled Generative Adversarial Networks
• GAN Augmentation: Augmenting Training Data using Generative Adversarial Networks
• On Convergence and Stability of GANs
• Local SGD Converges Fast and Communicates Little
• MD-GAN: Multi-Discriminator Generative Adversarial Networks for Distributed Datasets
• Cooperative SGD: A unified Framework for the Design and Analysis of Communication-Efficient SGD Algorithms
• Convergence rate of linear two-time-scale stochastic approximation
• Gradient descent GAN optimization is locally stable
• Generative Models for Effective ML on Private, Decentralized Datasets
• Decentralized Learning of Generative Adversarial Networks from Non-iid Data
• Federated AI lets a team imagine together: Federated Learning of GANs
• Convergence Analysis of Gradient-Based Learning with Non-Uniform Learning Rates in Non-Cooperative Multi-Agent Settings

Cited by in corpus (12)

• Adversarial Attacks Against Deep Generative Models on Data: A Survey
• Generative adversarial networks in time series: A survey and taxonomy
• Fed-TGAN: Federated Learning Framework for Synthesizing Tabular Data
• Generative Adversarial Networks: A Survey Towards Private and Secure Applications
• Local Stochastic Gradient Descent Ascent: Convergence Analysis and Communication Efficiency
• Bias-Free FedGAN: A Federated Approach to Generate Bias-Free Datasets
• Generation of Synthetic Electronic Health Records Using a Federated GAN
• Efficient Ring-topology Decentralized Federated Learning with Deep Generative Models for Industrial Artificial Intelligent
• Game of GANs: Game-Theoretical Models for Generative Adversarial Networks
• A New Distributed Method for Training Generative Adversarial Networks
• FedMM: Saddle Point Optimization for Federated Adversarial Domain Adaptation
• Federated Traffic Synthesizing and Classification Using Generative Adversarial Networks