Anycost GANs for Interactive Image Synthesis and Editing
arXiv:2103.03243
Abstract
Generative adversarial networks (GANs) have enabled photorealistic image synthesis and editing. However, due to the high computational cost of large-scale generators (e.g., StyleGAN2), it usually takes seconds to see the results of a single edit on edge devices, prohibiting interactive user experience. In this paper, we take inspirations from modern rendering software and propose Anycost GAN for interactive natural image editing. We train the Anycost GAN to support elastic resolutions and channels for faster image generation at versatile speeds. Running subsets of the full generator produce outputs that are perceptually similar to the full generator, making them a good proxy for preview. By using sampling-based multi-resolution training, adaptive-channel training, and a generator-conditioned discriminator, the anycost generator can be evaluated at various configurations while achieving better image quality compared to separately trained models. Furthermore, we develop new encoder training and latent code optimization techniques to encourage consistency between the different sub-generators during image projection. Anycost GAN can be executed at various cost budgets (up to 10x computation reduction) and adapt to a wide range of hardware and latency requirements. When deployed on desktop CPUs and edge devices, our model can provide perceptually similar previews at 6-12x speedup, enabling interactive image editing. The code and demo are publicly available: https://github.com/mit-han-lab/anycost-gan.
Accepted to CVPR 2021. The code and demo are available: https://github.com/mit-han-lab/anycost-gan
References in corpus (13)
- Distilling the Knowledge in a Neural Network
- MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
- GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
- Large Scale GAN Training for High Fidelity Natural Image Synthesis
- Self-Attention Generative Adversarial Networks
- DARTS: Differentiable Architecture Search
- LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
- AMC: AutoML for Model Compression and Acceleration on Mobile Devices
- Training Generative Adversarial Networks with Limited Data
- Compressing GANs using Knowledge Distillation
- Generative Adversarial Networks for Image and Video Synthesis: Algorithms and Applications
- Design Automation for Efficient Deep Learning Computing
- Self-Supervised GAN Compression