145 citations · 342 across the 27 of their papers we have counts for
28 papers
Putting People in Their Place: Affordance-Aware Human Insertion into Scenes
Sumith Kulal, Tim Brooks, Alex Aiken +5
We study the problem of inferring scene affordances by presenting a method for realistically inserting people into scenes. Given a scene image with a marked region and an image of…
Zero-shot Image-to-Image Translation
Gaurav Parmar, Krishna Kumar Singh, Richard Zhang +3
Large-scale text-to-image generative models have shown their remarkable ability to synthesize diverse and high-quality images. However, it is still challenging to directly apply th…
Enhanced Controllability of Diffusion Models via Feature Disentanglement and Realism-Enhanced Sampling Methods
Wonwoong Cho, Hareesh Ravi, Midhun Harikumar +5
As Diffusion Models have shown promising performance, a lot of efforts have been made to improve the controllability of Diffusion Models. However, how to train Diffusion Models to…
Modulating Pretrained Diffusion Models for Multimodal Image Synthesis
Cusuh Ham, James Hays, Jingwan Lu +3
We present multimodal conditioning modules (MCM) for enabling conditional image synthesis using pretrained diffusion models. Previous multimodal synthesis works rely on training ne…
Structure-Guided Image Completion with Image-level and Object-level Semantic Discriminators
Haitian Zheng, Zhe Lin, Jingwan Lu +8
Structure-guided image completion aims to inpaint a local region of an image according to an input guidance map from users. While such a task enables many practical applications fo…
Contrastive Learning for Diverse Disentangled Foreground Generation
Yuheng Li, Yijun Li, Jingwan Lu +3
We introduce a new method for diverse foreground generation with explicit control over various factors. Existing image inpainting based foreground generation methods often struggle…
UMFuse: Unified Multi View Fusion for Human Editing applications
Rishabh Jain, Mayur Hemani, Duygu Ceylan +4
Numerous pose-guided human editing methods have been explored by the vision community due to their extensive practical applications. However, most of these methods still use an ima…
VGFlow: Visibility guided Flow Network for Human Reposing
Rishabh Jain, Krishna Kumar Singh, Mayur Hemani +4
The task of human reposing involves generating a realistic image of a person standing in an arbitrary conceivable pose. There are multiple difficulties in generating perceptually a…
Spatially-Adaptive Multilayer Selection for GAN Inversion and Editing
Gaurav Parmar, Yijun Li, Jingwan Lu +3
Existing GAN inversion and editing methods work well for aligned objects with a clean background, such as portraits and animal faces, but often struggle for more difficult categori…
Learning Motion-Dependent Appearance for High-Fidelity Rendering of Dynamic Humans from a Single Camera
Jae Shin Yoon, Duygu Ceylan, Tuanfeng Y. Wang +4
Appearance of dressed humans undergoes a complex geometric transformation induced not only by the static pose but also by its dynamics, i.e., there exists a number of cloth geometr…
InsetGAN for Full-Body Image Generation
Anna Frühstück, Krishna Kumar Singh, Eli Shechtman +3
While GANs can produce photo-realistic images in ideal conditions for certain domains, the generation of full-body human images remains difficult due to the diversity of identities…
CM-GAN: Image Inpainting with Cascaded Modulation GAN and Object-Aware Training
Haitian Zheng, Zhe Lin, Jingwan Lu +7
Recent image inpainting methods have made great progress but often struggle to generate plausible image structures when dealing with large holes in complex images. This is partiall…
Collaging Class-specific GANs for Semantic Image Synthesis
Yuheng Li, Yijun Li, Jingwan Lu +3
We propose a new approach for high resolution semantic image synthesis. It consists of one base image generator and multiple class-specific generators. The base generator generates…
Pose with Style: Detail-Preserving Pose-Guided Image Synthesis with Conditional StyleGAN
Badour AlBahar, Jingwan Lu, Jimei Yang +3
We present an algorithm for re-rendering a person from a single image under arbitrary poses. Existing methods often have difficulties in hallucinating occluded contents photo-reali…
Few-shot Image Generation via Cross-domain Correspondence
Utkarsh Ojha, Yijun Li, Jingwan Lu +4
Training generative models, such as GANs, on a target domain containing limited examples (e.g., 10) can easily result in overfitting. In this work, we seek to utilize a large sourc…
IMAGINE: Image Synthesis by Image-Guided Model Inversion
Pei Wang, Yijun Li, Krishna Kumar Singh +2
We introduce an inversion based method, denoted as IMAge-Guided model INvErsion (IMAGINE), to generate high-quality and diverse images from only a single training sample. We levera…
Few-shot Image Generation with Elastic Weight Consolidation
Yijun Li, Richard Zhang, Jingwan Lu +1
Few-shot image generation seeks to generate more data of a given domain, with only few available training examples. As it is unreasonable to expect to fully infer the distribution…
Semantic Layout Manipulation with High-Resolution Sparse Attention
Haitian Zheng, Zhe Lin, Jingwan Lu +4
We tackle the problem of semantic image layout manipulation, which aims to manipulate an input image by editing its semantic label map. A core problem of this task is how to transf…
Unselfie: Translating Selfies to Neutral-pose Portraits in the Wild
Liqian Ma, Zhe Lin, Connelly Barnes +2
Due to the ubiquity of smartphones, it is popular to take photos of one's self, or "selfies." Such photos are convenient to take, because they do not require specialized equipment…
Modeling Artistic Workflows for Image Generation and Editing
Hung-Yu Tseng, Matthew Fisher, Jingwan Lu +3
People often create art by following an artistic workflow involving multiple stages that inform the overall design. If an artist wishes to modify an earlier decision, significant w…
Swapping Autoencoder for Deep Image Manipulation
Taesung Park, Jun-Yan Zhu, Oliver Wang +4
Deep generative models have become increasingly effective at producing realistic images from randomly sampled seeds, but using such models for controllable manipulation of existing…
Generative Tweening: Long-term Inbetweening of 3D Human Motions
Yi Zhou, Jingwan Lu, Connelly Barnes +3
The ability to generate complex and realistic human body animations at scale, while following specific artistic constraints, has been a fundamental goal for the game and animation…
AutoToon: Automatic Geometric Warping for Face Cartoon Generation
Julia Gong, Yannick Hold-Geoffroy, Jingwan Lu
Caricature, a type of exaggerated artistic portrait, amplifies the distinctive, yet nuanced traits of human faces. This task is typically left to artists, as it has proven difficul…
On the Continuity of Rotation Representations in Neural Networks
Yi Zhou, Connelly Barnes, Jingwan Lu +2
In neural networks, it is often desirable to work with various representations of the same space. For example, 3D rotations can be represented with quaternions or Euler angles. In…