A Survey on GAN Acceleration Using Memory Compression Technique
arXiv:2108.06626
Abstract
Since its invention, Generative adversarial networks (GANs) have shown outstanding results in many applications. Generative Adversarial Networks are powerful yet, resource-hungry deep-learning models. Their main difference from ordinary deep learning models is the nature of their output. For example, GAN output can be a whole image versus other models detecting objects or classifying images. Thus, the architecture and numeric precision of the network affect the quality and speed of the solution. Hence, accelerating GANs is pivotal. Accelerating GANs can be classified into three main tracks: (1) Memory compression, (2) Computation optimization, and (3) Data-flow optimization. Because data transfer is the main source of energy usage, memory compression leads to the most savings. Thus, in this paper, we survey memory compression techniques for CNN-Based GANs. Additionally, the paper summarizes opportunities and challenges in GANs acceleration and suggests open research problems to be further investigated.
arXiv comment21 pages, 17 figures
References in corpus (15)
- GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
- Knowledge Distillation: A Survey
- Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding
- Improved Training of Wasserstein GANs
- ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks
- A Survey of Model Compression and Acceleration for Deep Neural Networks
- Compressing Neural Networks with the Hashing Trick
- GANSynth: Adversarial Neural Audio Synthesis
- Adversarial Video Generation on Complex Datasets
- Compressing GANs using Knowledge Distillation
- Generative adversarial network-based glottal waveform model for statistical parametric speech synthesis
- QGAN: Quantized Generative Adversarial Networks
- P-KDGAN: Progressive Knowledge Distillation with GANs for One-class Novelty Detection
- Self-Supervised GAN Compression
- Content-Aware GAN Compression