CogView: Mastering Text-to-Image Generation via Transformers
arXiv:2105.13290
Abstract
Text-to-Image generation in the general domain has long been an open problem, which requires both a powerful generative model and cross-modal understanding. We propose CogView, a 4-billion-parameter Transformer with VQ-VAE tokenizer to advance this problem. We also demonstrate the finetuning strategies for various downstream tasks, e.g. style learning, super-resolution, text-image ranking and fashion design, and methods to stabilize pretraining, e.g. eliminating NaN losses. CogView achieves the state-of-the-art FID on the blurred MS COCO dataset, outperforming previous GAN-based models and a recent similar work DALL-E.
arXiv commentto appear in NeurIPS 2021
References in corpus (11)
- Auto-Encoding Variational Bayes
- GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
- Semantics derived automatically from language corpora contain human-like biases
- Neural Discrete Representation Learning
- Generative Adversarial Text to Image Synthesis
- Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
- SGDR: Stochastic Gradient Descent with Warm Restarts
- Pixel Recurrent Neural Networks
- Zero-Shot Text-to-Image Generation
- Lagging Inference Networks and Posterior Collapse in Variational Autoencoders
- On Layer Normalization in the Transformer Architecture
Cited by in corpus (28)
- A Survey on Visual Transformer
- WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing
- Blended Diffusion for Text-driven Editing of Natural Images
- Blended Latent Diffusion
- Compute Trends Across Three Eras of Machine Learning
- FILIP: Fine-grained Interactive Language-Image Pre-Training
- SpaText: Spatio-Textual Representation for Controllable Image Generation
- DiffEdit: Diffusion-based semantic image editing with mask guidance
- Improving Video-Text Retrieval by Multi-Stream Corpus Alignment and Dual Softmax Loss
- A Survey of AI Text-to-Image and AI Text-to-Video Generators
- Exploring the Roles of Large Language Models in Reshaping Transportation Systems: A Survey, Framework, and Roadmap
- Guided Image Synthesis via Initial Image Editing in Diffusion Model
- Vector Quantized Diffusion Model for Text-to-Image Synthesis
- Unifying Multimodal Transformer for Bi-directional Image and Text Generation
- Not Only Generative Art: Stable Diffusion for Content-Style Disentanglement in Art Analysis
- NormFormer: Improved Transformer Pretraining with Extra Normalization
- ARMANI: Part-level Garment-Text Alignment for Unified Cross-Modal Fashion Design
- Taming Visually Guided Sound Generation
- M6-10T: A Sharing-Delinking Paradigm for Efficient Multi-Trillion Parameter Pretraining
- Adma-GAN: Attribute-Driven Memory Augmented GANs for Text-to-Image Generation
- Pre-Trained Models: Past, Present and Future
- Can Artificial Intelligence Reconstruct Ancient Mosaics?
- Translation-Enhanced Multilingual Text-to-Image Generation
- F3-Pruning: A Training-Free and Generalized Pruning Strategy towards Faster and Finer Text-to-Video Synthesis
- Intelligent Generation of Graphical Game Assets: A Conceptual Framework and Systematic Review of the State of the Art
- Image-Text Out-Of-Context Detection Using Synthetic Multimodal Misinformation
- Offline Evaluation of Set-Based Text-to-Image Generation
- TPA3D: Triplane Attention for Fast Text-to-3D Generation