CariGAN: Caricature Generation through Weakly Paired Adversarial Learning
arXiv:1811.00445
Abstract
Caricature generation is an interesting yet challenging task. The primary goal is to generate plausible caricatures with reasonable exaggerations given face images. Conventional caricature generation approaches mainly use low-level geometric transformations such as image warping to generate exaggerated images, which lack richness and diversity in terms of content and style. The recent progress in generative adversarial networks (GANs) makes it possible to learn an image-to-image transformation from data, so that richer contents and styles can be generated. However, directly applying the GAN-based models to this task leads to unsatisfactory results because there is a large variance in the caricature distribution. Moreover, some models require strictly paired training data which largely limits their usage scenarios. In this paper, we propose CariGAN overcome these problems. Instead of training on paired data, CariGAN learns transformations only from weakly paired images. Specifically, to enforce reasonable exaggeration and facial deformation, facial landmarks are adopted as an additional condition to constrain the generated image. Furthermore, an attention mechanism is introduced to encourage our model to focus on the key facial parts so that more vivid details in these regions can be generated. Finally, a Diversity Loss is proposed to encourage the model to produce diverse results to help alleviate the `mode collapse' problem of the conventional GAN-based models. Extensive experiments on a new large-scale `WebCaricature' dataset show that the proposed CariGAN can generate more plausible caricatures with larger diversity compared with the state-of-the-art models.
arXiv comment12
References in corpus (21)
- Auto-Encoding Variational Bayes
- Conditional Generative Adversarial Nets
- Spectral Normalization for Generative Adversarial Networks
- Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks
- Empirical Evaluation of Rectified Activations in Convolutional Network
- Generative Adversarial Text to Image Synthesis
- NIPS 2016 Tutorial: Generative Adversarial Networks
- InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets
- Which Training Methods for GANs do actually Converge?
- BEGAN: Boundary Equilibrium Generative Adversarial Networks
- Adversarial Feature Learning
- Autoencoding beyond pixels using a learned similarity metric
- Deep multi-scale video prediction beyond mean square error
- Unsupervised and Semi-supervised Learning with Categorical Generative Adversarial Networks
- DualGAN: Unsupervised Dual Learning for Image-to-Image Translation
- Multimodal Unsupervised Image-to-Image Translation
- Stochastic Adversarial Video Prediction
- WebCaricature: a benchmark for caricature recognition
- Show, Adapt and Tell: Adversarial Training of Cross-domain Image Captioner
- Learning to Generate Time-Lapse Videos Using Multi-Stage Dynamic Generative Adversarial Networks
- Unpaired Photo-to-Caricature Translation on Faces in the Wild
Cited by in corpus (12)
- 3D-CariGAN: An End-to-End Solution to 3D Caricature Generation from Face Photos
- MW-GAN: Multi-Warping GAN for Caricature Generation with Multi-Style Geometric Exaggeration
- Modeling Caricature Expressions by 3D Blendshape and Dynamic Texture
- Face Completion with Semantic Knowledge and Collaborative Adversarial Learning
- Geometric Style Transfer
- WarpGAN: Automatic Caricature Generation
- Learning to Caricature via Semantic Shape Transform
- MangaGAN: Unpaired Photo-to-Manga Translation Based on The Methodology of Manga Drawing
- Fine-grained Image-to-Image Transformation towards Visual Recognition
- Exemplar-Based 3D Portrait Stylization
- Non-Parametric Neural Style Transfer
- Unsupervised Contrastive Photo-to-Caricature Translation based on Auto-distortion