Synthesizing Photorealistic Images with Deep Generative Learning

Abstract

The goal of this thesis is to present my research contributions towards solving various visual synthesis and generation tasks, comprising image translation, image completion, and completed scene decomposition. This thesis consists of five pieces of work, each of which presents a new learning-based approach for synthesizing images with plausible content as well as visually realistic appearance. Each work demonstrates the superiority of the proposed approach on image synthesis, with some further contributing to other tasks, such as depth estimation.

arXiv commentPhD thesis

References in corpus (79)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• Auto-Encoding Variational Bayes
• Neural Machine Translation by Jointly Learning to Align and Translate
• Conditional Generative Adversarial Nets
• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• A Simple Framework for Contrastive Learning of Visual Representations
• Representation Learning with Contrastive Predictive Coding
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Fully Convolutional Networks for Semantic Segmentation
• Neural Discrete Representation Learning
• Unsupervised Domain Adaptation by Backpropagation
• Self-Attention Generative Adversarial Networks
• Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks
• Deformable DETR: Deformable Transformers for End-to-End Object Detection
• Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
• Learning deep representations by mutual information estimation and maximization
• Multi-Scale Context Aggregation by Dilated Convolutions
• Improved Training of Wasserstein GANs
• NICE: Non-linear Independent Components Estimation
• Learning Depth from Single Monocular Images Using Deep Convolutional Neural Fields
• Toward Multimodal Image-to-Image Translation
• Momentum Contrast for Unsupervised Visual Representation Learning
• Data-Efficient Image Recognition with Contrastive Predictive Coding
• The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
• Which Training Methods for GANs do actually Converge?
• Learning to Discover Cross-Domain Relations with Generative Adversarial Networks
• Learning to Segment Object Candidates
• Learning Representations by Maximizing Mutual Information Across Views
• CyCADA: Cycle-Consistent Adversarial Domain Adaptation
• Unsupervised Image-to-Image Translation Networks
• Visual Transformers: Token-based Image Representation and Processing for Computer Vision
• NVAE: A Deep Hierarchical Variational Autoencoder
• DualGAN: Unsupervised Dual Learning for Image-to-Image Translation
• High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs
• Multimodal Unsupervised Image-to-Image Translation
• Depth Extraction from Video Using Non-parametric Sampling
• Unsupervised Attention-guided Image to Image Translation
• Semantic Image Synthesis with Spatially-Adaptive Normalization
• Image Inpainting via Generative Multi-column Convolutional Neural Networks
• Variational Approaches for Auto-Encoding Generative Adversarial Networks
• Generative Image Inpainting with Contextual Attention
• Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers
• DRIT++: Diverse Image-to-Image Translation via Disentangled Representations
• Glow: Generative Flow with Invertible 1x1 Convolutions
• Hybrid Task Cascade for Instance Segmentation
• Generating Images with Perceptual Similarity Metrics based on Deep Networks
• Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images
• Contrastive Learning for Unpaired Image-to-Image Translation
• SPG-Net: Segmentation Prediction and Guidance Network for Image Inpainting
• Coupled Generative Adversarial Networks
• StarGAN v2: Diverse Image Synthesis for Multiple Domains
• Photographic Image Synthesis with Cascaded Refinement Networks
• Semantic Image Inpainting with Deep Generative Models
• Learning to Refine Object Segments
• CVAE-GAN: Fine-Grained Image Generation through Asymmetric Training
• One-Sided Unsupervised Domain Mapping
• Reliable Fidelity and Diversity Metrics for Generative Models
• Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer
• Shift-Net: Image Inpainting via Deep Feature Rearrangement
• Estimating Depth from Monocular Images as Classification Using Deep Fully Convolutional Residual Networks
• Photorealistic Style Transfer via Wavelet Transforms
• Contextual Residual Aggregation for Ultra High-Resolution Image Inpainting
• High Resolution Face Completion with Multiple Controllable Attributes via Fully End-to-End Progressive Generative Adversarial Networks
• Rethinking Image Inpainting via a Mutual Encoder-Decoder with Feature Equalizations
• Learning Image Representations by Completing Damaged Jigsaw Puzzles
• FaceShop: Deep Sketch-based Face Image Editing
• Geometry-Consistent Generative Adversarial Networks for One-Sided Unsupervised Domain Mapping
• UnrealCV: Connecting Computer Vision to Unreal Engine
• The Spatially-Correlative Loss for Various Image Translation Tasks
• Geometry-Aware Symmetric Domain Adaptation for Monocular Depth Estimation
• High-Resolution Image Inpainting with Iterative Confidence Feedback and Guided Upsampling
• TSIT: A Simple and Versatile Framework for Image-to-Image Translation
• Visualizing the Invisible: Occluded Vehicle Segmentation and Recovery
• Learning Object-Compositional Neural Radiance Field for Editable Scene Rendering
• Attention-GAN for Object Transfiguration in Wild Images
• S2R-DepthNet: Learning a Generalizable Depth-specific Structural Representation
• Spherical Image Generation from a Single Normal Field of View Image by Considering Scene Symmetry
• Visiting the Invisible: Layer-by-Layer Completed Scene Decomposition