301 citations · 1k across the 20 of their papers we have counts for
20 papers
Edify Image: High-Quality Image Generation with Pixel Space Laplacian Diffusion Models
NVIDIA, :, Yuval Atzmon +29
We introduce Edify Image, a family of diffusion models capable of generating photorealistic image content with pixel-perfect accuracy. Edify Image utilizes cascaded pixel-space dif…
JeDi: Joint-Image Diffusion Models for Finetuning-Free Personalized Text-to-Image Generation
Yu Zeng, Vishal M. Patel, Haochen Wang +4
Personalized text-to-image generation models enable users to create images that depict their individual possessions in diverse scenes, finding applications in various domains. To a…
DreamPose: Fashion Image-to-Video Synthesis via Stable Diffusion
Johanna Karras, Aleksander Holynski, Ting-Chun Wang +1
We present DreamPose, a diffusion-based method for generating animated fashion videos from still images. Given an image and a sequence of human body poses, our method synthesizes a…
SPACE: Speech-driven Portrait Animation with Controllable Expression
Siddharth Gururani, Arun Mallya, Ting-Chun Wang +2
Animating portraits using speech has received growing attention in recent years, with various creative and practical use cases. An ideal generated video should have good lip sync w…
Implicit Warping for Animation with Image Sets
Arun Mallya, Ting-Chun Wang, Ming-Yu Liu
We present a new implicit warping framework for image animation using sets of source images through the transfer of the motion of a driving video. A single cross- modal attention l…
Learning to Relight Portrait Images via a Virtual Light Stage and Synthetic-to-Real Adaptation
Yu-Ying Yeh, Koki Nagano, Sameh Khamis +3
Given a portrait image of a person and an environment map of the target lighting, portrait relighting aims to re-illuminate the person in the image as if the person appeared in an…
Generating Long Videos of Dynamic Scenes
Tim Brooks, Janne Hellsten, Miika Aittala +6
We present a video generation model that accurately reproduces object motion, changes in camera viewpoint, and new content that arises over time. Existing video generation methods…
Multimodal Conditional Image Synthesis with Product-of-Experts GANs
Xun Huang, Arun Mallya, Ting-Chun Wang +1
Existing conditional image synthesis frameworks generate images based on user inputs in a single modality, such as text, segmentation, sketch, or style reference. They are often un…
One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing
Ting-Chun Wang, Arun Mallya, Ming-Yu Liu
We propose a neural talking-head video synthesis model and demonstrate its application to video conferencing. Our model learns to synthesize a talking-head video using a source ima…
Generative Adversarial Networks for Image and Video Synthesis: Algorithms and Applications
Ming-Yu Liu, Xun Huang, Jiahui Yu +2
The generative adversarial network (GAN) framework has emerged as a powerful tool for various image and video synthesis tasks, allowing the synthesis of visual content in an uncond…
World-Consistent Video-to-Video Synthesis
Arun Mallya, Ting-Chun Wang, Karan Sapra +1
Video-to-video synthesis (vid2vid) aims for converting high-level semantic inputs to photorealistic videos. While existing vid2vid methods can achieve short-term temporal consisten…
Transposer: Universal Texture Synthesis Using Feature Maps as Transposed Convolution Filter
Guilin Liu, Rohan Taori, Ting-Chun Wang +6
Conventional CNNs for texture synthesis consist of a sequence of (de)-convolution and up/down-sampling layers, where each layer operates locally and lacks the ability to capture th…
Dancing to Music
Hsin-Ying Lee, Xiaodong Yang, Ming-Yu Liu +4
Dancing to music is an instinctive move by humans. Learning to model the music-to-dance generation process is, however, a challenging problem. It requires significant efforts to me…
Few-shot Video-to-Video Synthesis
Ting-Chun Wang, Ming-Yu Liu, Andrew Tao +3
Video-to-video synthesis (vid2vid) aims at converting an input semantic video, such as videos of human poses or segmentation masks, to an output photorealistic video. While the sta…
Semantic Image Synthesis with Spatially-Adaptive Normalization
Taesung Park, Ming-Yu Liu, Ting-Chun Wang +1
We propose spatially-adaptive normalization, a simple but effective layer for synthesizing photorealistic images given an input semantic layout. Previous methods directly feed the…
Partial Convolution based Padding
Guilin Liu, Kevin J. Shih, Ting-Chun Wang +5
In this paper, we present a simple yet effective padding scheme that can be used as a drop-in module for existing convolutional neural networks. We call it partial convolution base…
Video-to-Video Synthesis
Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu +4
We study the problem of video-to-video synthesis, whose goal is to learn a mapping function from an input source video (e.g., a sequence of semantic segmentation masks) to an outpu…
Domain Stylization: A Strong, Simple Baseline for Synthetic to Real Image Domain Adaptation
Aysegul Dundar, Ming-Yu Liu, Ting-Chun Wang +2
Deep neural networks have largely failed to effectively utilize synthetic data when applied to real images due to the covariate shift problem. In this paper, we show that by applyi…
Image Inpainting for Irregular Holes Using Partial Convolutions
Guilin Liu, Fitsum A. Reda, Kevin J. Shih +3
Existing deep learning based image inpainting methods use a standard convolutional network over the corrupted image, using convolutional filter responses conditioned on both valid…
High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs
Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu +3
We present a new method for synthesizing high-resolution photo-realistic images from semantic label maps using conditional generative adversarial networks (conditional GANs). Condi…