Text-to-image Synthesis via Symmetrical Distillation Networks
arXiv:1808.06801
Abstract
Text-to-image synthesis aims to automatically generate images according to text descriptions given by users, which is a highly challenging task. The main issues of text-to-image synthesis lie in two gaps: the heterogeneous and homogeneous gaps. The heterogeneous gap is between the high-level concepts of text descriptions and the pixel-level contents of images, while the homogeneous gap exists between synthetic image distributions and real image distributions. For addressing these problems, we exploit the excellent capability of generic discriminative models (e.g. VGG19), which can guide the training process of a new generative model on multiple levels to bridge the two gaps. The high-level representations can teach the generative model to extract necessary visual information from text descriptions, which can bridge the heterogeneous gap. The mid-level and low-level representations can lead it to learn structures and details of images respectively, which relieves the homogeneous gap. Therefore, we propose Symmetrical Distillation Networks (SDN) composed of a source discriminative model as "teacher" and a target generative model as "student". The target generative model has a symmetrical structure with the source discriminative model, in order to transfer hierarchical knowledge accessibly. Moreover, we decompose the training process into two stages with different distillation paradigms for promoting the performance of the target generative model. Experiments on two widely-used datasets are conducted to verify the effectiveness of our proposed SDN.
arXiv comment9 pages, accepted as an oral paper of ACM Multimedia 2018
References in corpus (12)
- Auto-Encoding Variational Bayes
- Distilling the Knowledge in a Neural Network
- Generative Adversarial Text to Image Synthesis
- Conditional Image Generation with PixelCNN Decoders
- Pixel Recurrent Neural Networks
- Improved Training of Wasserstein GANs
- Improved Techniques for Training GANs
- BEGAN: Boundary Equilibrium Generative Adversarial Networks
- Pose Guided Person Image Generation
- Synthesizing the preferred inputs for neurons in neural networks via deep generator networks
- Learning Deep Representations of Fine-grained Visual Descriptions
- Augmenting Supervised Neural Networks with Unsupervised Objectives for Large-scale Image Classification