MD-GAN: Multi-Discriminator Generative Adversarial Networks for Distributed Datasets

Abstract

A recent technical breakthrough in the domain of machine learning is the discovery and the multiple applications of Generative Adversarial Networks (GANs). Those generative models are computationally demanding, as a GAN is composed of two deep neural networks, and because it trains on large datasets. A GAN is generally trained on a single server. In this paper, we address the problem of distributing GANs so that they are able to train over datasets that are spread on multiple workers. MD-GAN is exposed as the first solution for this problem: we propose a novel learning procedure for GANs so that they fit this distributed setup. We then compare the performance of MD-GAN to an adapted version of Federated Learning to GANs, using the MNIST and CIFAR10 datasets. MD-GAN exhibits a reduction by a factor of two of the learning complexity on each worker node, while providing better performances than federated learning on both datasets. We finally discuss the practical implications of distributing GANs.

To be published in IPDPS 2019: the 33rd IEEE International Parallel & Distributed Processing Symposium

References in corpus (18)

• TensorFlow: A system for large-scale machine learning
• Generative Adversarial Networks
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Federated Learning: Strategies for Improving Communication Efficiency
• Generative Adversarial Text to Image Synthesis
• Conditional Image Synthesis With Auxiliary Classifier GANs
• Generating Videos with Scene Dynamics
• Revisiting Distributed Synchronous SGD
• Invertible Conditional GANs for image editing
• Towards Principled Methods for Training Generative Adversarial Networks
• AdaGAN: Boosting Generative Models
• Generative Multi-Adversarial Networks
• Multi-Generator Generative Adversarial Nets
• Towards Geo-Distributed Machine Learning
• Ensembles of Generative Adversarial Networks
• Gossip training for deep learning
• Generative Adversarial Parallelization
• Style Transfer Generative Adversarial Networks: Learning to Play Chess Differently

Cited by in corpus (18)

• FedML: A Research Library and Benchmark for Federated Machine Learning
• Adversarial Attacks Against Deep Generative Models on Data: A Survey
• FedCG: Leverage Conditional GAN for Protecting Privacy and Maintaining Competitive Performance in Federated Learning
• Generative adversarial networks in time series: A survey and taxonomy
• FedGAN: Federated Generative Adversarial Networks for Distributed Data
• Fed-TGAN: Federated Learning Framework for Synthesizing Tabular Data
• Decentralized Learning of Generative Adversarial Networks from Non-iid Data
• Generative Adversarial Networks: A Survey Towards Private and Secure Applications
• A Systematic Literature Review on Federated Learning: From A Model Quality Perspective
• Distributed Conditional Generative Adversarial Networks (GANs) for Data-Driven Millimeter Wave Communications in UAV Networks
• Training Federated GANs with Theoretical Guarantees: A Universal Aggregation Approach
• Generative Adversarial Network with Multi-Branch Discriminator for Cross-Species Image-to-Image Translation
• Learn distributed GAN with Temporary Discriminators
• One-Shot Image-to-Image Translation via Part-Global Learning with a Multi-adversarial Framework
• A New Distributed Method for Training Generative Adversarial Networks
• Game of GANs: Game-Theoretical Models for Generative Adversarial Networks
• Federated Traffic Synthesizing and Classification Using Generative Adversarial Networks
• HydraGAN A Multi-head, Multi-objective Approach to Synthetic Data Generation