99 citations · 217 across the 24 of their papers we have counts for
24 papers
Photoswap: Personalized Subject Swapping in Images
Jing Gu, Yilin Wang, Nanxuan Zhao +8
In an era where images and visual content dominate our digital landscape, the ability to manipulate and personalize these images has become a necessity. Envision seamlessly substit…
DualVector: Unsupervised Vector Font Synthesis with Dual-Part Representation
Ying-Tian Liu, Zhifei Zhang, Yuan-Chen Guo +3
Automatic generation of fonts can be an important aid to typeface design. Many current approaches regard glyphs as pixelated images, which present artifacts when scaling and inevit…
Improving Diffusion Models for Scene Text Editing with Dual Encoders
Jiabao Ji, Guanhua Zhang, Zhaowen Wang +4
Scene text editing is a challenging task that involves modifying or inserting specified texts in an image while maintaining its natural and realistic appearance. Most previous appr…
TopNet: Transformer-based Object Placement Network for Image Compositing
Sijie Zhu, Zhe Lin, Scott Cohen +3
We investigate the problem of automatically placing an object into a background image for image compositing. Given a background image and a segmented object, the goal is to train a…
Modulating Pretrained Diffusion Models for Multimodal Image Synthesis
Cusuh Ham, James Hays, Jingwan Lu +3
We present multimodal conditioning modules (MCM) for enabling conditional image synthesis using pretrained diffusion models. Previous multimodal synthesis works rely on training ne…
SmartBrush: Text and Shape Guided Object Inpainting with Diffusion Model
Shaoan Xie, Zhifei Zhang, Zhe Lin +2
Generic image inpainting aims to complete a corrupted image by borrowing surrounding information, which barely generates novel content. By contrast, multi-modal inpainting provides…
ObjectStitch: Generative Object Compositing
Yizhi Song, Zhifei Zhang, Zhe Lin +5
Object compositing based on 2D images is a challenging problem since it typically involves multiple processing stages such as color harmonization, geometry correction and shadow ge…
Toward Understanding WordArt: Corner-Guided Transformer for Scene Text Recognition
Xudong Xie, Ling Fu, Zhifei Zhang +2
Artistic text recognition is an extremely challenging task with a wide range of applications. However, current scene text recognition methods mainly focus on irregular text while h…
Vanilla Feature Distillation for Improving the Accuracy-Robustness Trade-Off in Adversarial Training
Guodong Cao, Zhibo Wang, Xiaowei Dong +4
Adversarial training has been widely explored for mitigating attacks against deep models. However, most existing works are still trapped in the dilemma between higher accuracy and…
SImProv: Scalable Image Provenance Framework for Robust Content Attribution
Alexander Black, Tu Bui, Simon Jenni +3
We present SImProv - a scalable image provenance framework to match a query image back to a trusted database of originals and identify possible manipulations on the query. SImProv…
GALA: Toward Geometry-and-Lighting-Aware Object Search for Compositing
Sijie Zhu, Zhe Lin, Scott Cohen +3
Compositing-aware object search aims to find the most compatible objects for compositing given a background image and a query bounding box. Previous works focus on learning compati…
Fairness-aware Adversarial Perturbation Towards Bias Mitigation for Deployed Deep Models
Zhibo Wang, Xiaowei Dong, Henry Xue +4
Prioritizing fairness is of central importance in artificial intelligence (AI) systems, especially for those societal applications, e.g., hiring systems should recommend applicants…
Font Completion and Manipulation by Cycling Between Multi-Modality Representations
Ye Yuan, Wuyang Chen, Zhaowen Wang +4
Generating font glyphs of consistent style from one or a few reference glyphs, i.e., font completion, is an important task in topographical design. As the problem is more well-defi…
Feature Importance-aware Transferable Adversarial Attacks
Zhibo Wang, Hengchang Guo, Zhifei Zhang +3
Transferability of adversarial examples is of central importance for attacking an unknown model, which facilitates adversarial attacks in more practical scenarios, e.g., black-box…
A Multi-Implicit Neural Representation for Fonts
Pradyumna Reddy, Zhifei Zhang, Matthew Fisher +3
Fonts are ubiquitous across documents and come in a variety of styles. They are either represented in a native vector format or rasterized to produce fixed resolution images. In th…
Rethinking Text Segmentation: A Novel Dataset and A Text-Specific Refinement Approach
Xingqian Xu, Zhifei Zhang, Zhaowen Wang +3
Text segmentation is a prerequisite in many real-world text-related tasks, e.g., text style transfer, and scene text removal. However, facing the lack of high-quality datasets and…
Texture Hallucination for Large-Factor Painting Super-Resolution
Yulun Zhang, Zhifei Zhang, Stephen DiVerdi +3
We aim to super-resolve digital paintings, synthesizing realistic details from high-resolution reference painting materials for very large scaling factors (e.g., 8X, 16X). However,…
Image Super-Resolution by Neural Texture Transfer
Zhifei Zhang, Zhaowen Wang, Zhe Lin +1
Due to the significant information loss in low-resolution (LR) images, it has become extremely challenging to further advance the state-of-the-art of single image super-resolution…
Beyond Inferring Class Representatives: User-Level Privacy Leakage From Federated Learning
Zhibo Wang, Mengkai Song, Zhifei Zhang +3
Federated learning, i.e., a mobile edge computing framework for deep learning, is a recent advance in privacy-preserving machine learning, where the model is trained in a decentral…
Reference-Conditioned Super-Resolution by Neural Texture Transfer
Zhifei Zhang, Zhaowen Wang, Zhe Lin +1
With the recent advancement in deep learning, we have witnessed a great progress in single image super-resolution. However, due to the significant information loss of the image dow…
Decoupled Learning for Conditional Adversarial Networks
Zhifei Zhang, Yang Song, Hairong Qi
Incorporating encoding-decoding nets with adversarial nets has been widely adopted in image generation tasks. We observe that the state-of-the-art achievements were obtained by car…
r-BTN: Cross-domain Face Composite and Synthesis from Limited Facial Patches
Yang Song, Zhifei Zhang, Hairong Qi
We start by asking an interesting yet challenging question, "If an eyewitness can only recall the eye features of the suspect, such that the forensic artist can only produce a sket…
Age Progression/Regression by Conditional Adversarial Autoencoder
Zhifei Zhang, Yang Song, Hairong Qi
"If I provide you a face image of mine (without telling you the actual age when I took the picture) and a large amount of face images that I crawled (containing labeled faces of di…
Derivative Delay Embedding: Online Modeling of Streaming Time Series
Zhifei Zhang, Yang Song, Wei Wang +1
The staggering amount of streaming time series coming from the real world calls for more efficient and effective online modeling solution. For time series modeling, most existing w…