201 citations · 339 across the 24 of their papers we have counts for
25 papers
UniSER: A Foundation Model for Unified Soft Effects Removal
Jingdong Zhang, Lingzhi Zhang, Qing Liu +12
Digital images are often degraded by soft effects such as lens flare, haze, shadows, and reflections, which reduce aesthetics even though the underlying pixels remain partially vis…
Fine-grained Defocus Blur Control for Generative Image Models
Ayush Shrivastava, Connelly Barnes, Xuaner Zhang +4
Current text-to-image diffusion models excel at generating diverse, high-quality images, yet they struggle to incorporate fine-grained camera metadata such as precise aperture sett…
ZipIR: Latent Pyramid Diffusion Transformer for High-Resolution Image Restoration
Yongsheng Yu, Haitian Zheng, Zhifei Zhang +7
Recent progress in generative models has significantly improved image restoration capabilities, particularly through powerful diffusion models that offer remarkable recovery of sem…
Layer- and Timestep-Adaptive Differentiable Token Compression Ratios for Efficient Diffusion Transformers
Haoran You, Connelly Barnes, Yuqian Zhou +10
Diffusion Transformers (DiTs) have achieved state-of-the-art (SOTA) image generation quality but suffer from high latency and memory inefficiency, making them difficult to deploy o…
Distilling Diffusion Models into Conditional GANs
Minguk Kang, Richard Zhang, Connelly Barnes +6
We propose a method to distill a complex multistep diffusion model into a single-step conditional GAN student model, dramatically accelerating inference, while preserving image qua…
Perceptual Artifacts Localization for Image Synthesis Tasks
Lingzhi Zhang, Zhengjie Xu, Connelly Barnes +7
Recent advancements in deep generative models have facilitated the creation of photo-realistic images across various tasks. However, these generated images often exhibit perceptual…
SimpSON: Simplifying Photo Cleanup with Single-Click Distracting Object Segmentation Network
Chuong Huynh, Yuqian Zhou, Zhe Lin +4
In photo editing, it is common practice to remove visual distractions to improve the overall image quality and highlight the primary subject. However, manually selecting and removi…
Automatic High Resolution Wire Segmentation and Removal
Mang Tik Chiu, Xuaner Zhang, Zijun Wei +7
Wires and powerlines are common visual distractions that often undermine the aesthetics of photographs. The manual process of precisely segmenting and removing them is extremely te…
Structure-Guided Image Completion with Image-level and Object-level Semantic Discriminators
Haitian Zheng, Zhe Lin, Jingwan Lu +8
Structure-guided image completion aims to inpaint a local region of an image according to an input guidance map from users. While such a task enables many practical applications fo…
Deep 360 Optical Flow Estimation Based on Multi-Projection Fusion
Yiheng Li, Connelly Barnes, Kun Huang +1
Optical flow computation is essential in the early stages of the video processing pipeline. This paper focuses on a less explored problem in this area, the 360 optical flow…
CM-GAN: Image Inpainting with Cascaded Modulation GAN and Object-Aware Training
Haitian Zheng, Zhe Lin, Jingwan Lu +7
Recent image inpainting methods have made great progress but often struggle to generate plausible image structures when dealing with large holes in complex images. This is partiall…
GeoFill: Reference-Based Image Inpainting with Better Geometric Understanding
Yunhan Zhao, Connelly Barnes, Yuqian Zhou +3
Reference-guided image inpainting restores image pixels by leveraging the content from another single reference image. The primary challenge is how to precisely place the pixels fr…
Modulated Periodic Activations for Generalizable Local Functional Representations
Ishit Mehta, Michaël Gharbi, Connelly Barnes +3
Multi-Layer Perceptrons (MLPs) make powerful functional representations for sampling and reconstruction problems involving low-dimensional signals like images,shapes and light fiel…
TransFill: Reference-guided Image Inpainting by Merging Multiple Color and Spatial Transformations
Yuqian Zhou, Connelly Barnes, Eli Shechtman +1
Image inpainting is the task of plausibly restoring missing pixels within a hole region that is to be removed from a target image. Most existing technologies exploit patch similari…
Learning from Shader Program Traces
Yuting Yang, Connelly Barnes, Adam Finkelstein
Deep learning for image processing typically treats input imagery as pixels in some color space. This paper proposes instead to learn from program traces of procedural fragment sha…
Unselfie: Translating Selfies to Neutral-pose Portraits in the Wild
Liqian Ma, Zhe Lin, Connelly Barnes +2
Due to the ubiquity of smartphones, it is popular to take photos of one's self, or "selfies." Such photos are convenient to take, because they do not require specialized equipment…
Generative Tweening: Long-term Inbetweening of 3D Human Motions
Yi Zhou, Jingwan Lu, Connelly Barnes +3
The ability to generate complex and realistic human body animations at scale, while following specific artistic constraints, has been a fundamental goal for the game and animation…
Image Morphing with Perceptual Constraints and STN Alignment
Noa Fish, Richard Zhang, Lilach Perry +3
In image morphing, a sequence of plausible frames are synthesized and composited together to form a smooth transformation between given instances. Intermediates must remain faithfu…
Foreground-aware Image Inpainting
Wei Xiong, Jiahui Yu, Zhe Lin +4
Existing image inpainting methods typically fill holes by borrowing information from surrounding pixels. They often produce unsatisfactory results when the holes overlap with or to…
Texture Mixer: A Network for Controllable Synthesis and Interpolation of Texture
Ning Yu, Connelly Barnes, Eli Shechtman +2
This paper addresses the problem of interpolating visual textures. We formulate this problem by requiring (1) by-example controllability and (2) realistic and smooth interpolation…
On the Continuity of Rotation Representations in Neural Networks
Yi Zhou, Connelly Barnes, Jingwan Lu +2
In neural networks, it is often desirable to work with various representations of the same space. For example, 3D rotations can be represented with quaternions or Euler angles. In…
Approximate Program Smoothing Using Mean-Variance Statistics, with Application to Procedural Shader Bandlimiting
Yuting Yang, Connelly Barnes
This paper introduces a general method to approximate the convolution of an arbitrary program with a Gaussian kernel. This process has the effect of smoothing out a program. Our co…
Where and Who? Automatic Semantic-Aware Person Composition
Fuwen Tan, Crispin Bernier, Benjamin Cohen +2
Image compositing is a method used to generate realistic yet fake imagery by inserting contents from one image to another. Previous work in compositing has focused on improving app…
Stable and Controllable Neural Texture Synthesis and Style Transfer Using Histogram Losses
Eric Risser, Pierre Wilmot, Connelly Barnes
Recently, methods have been proposed that perform texture synthesis and style transfer by using convolutional neural networks (e.g. Gatys et al. [2015,2016]). These methods are exc…