40 citations · 158 across the 21 of their papers we have counts for
22 papers
Incorporating Pre-trained Diffusion Models in Solving the Schrödinger Bridge Problem
Zhicong Tang, Tiankai Hang, Shuyang Gu +2
This paper aims to unify Score-based Generative Models (SGMs), also known as Diffusion models, and the Schrödinger Bridge (SB) problem through three reparameterization techniques:…
DesignDiffusion: High-Quality Text-to-Design Image Generation with Diffusion Models
Zhendong Wang, Jianmin Bao, Shuyang Gu +3
In this paper, we present DesignDiffusion, a simple yet effective framework for the novel task of synthesizing design images from textual descriptions. A primary challenge lies in…
Improved Noise Schedule for Diffusion Training
Tiankai Hang, Shuyang Gu, Xin Geng +1
Diffusion models have emerged as the de facto choice for generating high-quality visual signals across various domains. However, training a single model to predict noise across var…
FontStudio: Shape-Adaptive Diffusion Model for Coherent and Consistent Font Effect Generation
Xinzhi Mu, Li Chen, Bohan Chen +5
Recently, the application of modern diffusion-based text-to-image generation models for creating artistic fonts, traditionally the domain of professional designers, has garnered si…
Aesthetic Post-Training Diffusion Models from Generic Preferences with Step-by-step Preference Optimization
Zhanhao Liang, Yuhui Yuan, Shuyang Gu +5
Generating visually appealing images is fundamental to modern text-to-image generation models. A potential solution to better aesthetics is direct preference optimization (DPO), wh…
Simplified Diffusion Schrödinger Bridge
Zhicong Tang, Tiankai Hang, Shuyang Gu +2
This paper introduces a novel theoretical simplification of the Diffusion Schrödinger Bridge (DSB) that facilitates its unification with Score-based Generative Models (SGMs), addre…
CCA: Collaborative Competitive Agents for Image Editing
Tiankai Hang, Shuyang Gu, Dong Chen +2
This paper presents a novel generative model, Collaborative Competitive Agents (CCA), which leverages the capabilities of multiple Large Language Models (LLMs) based agents to exec…
VolumeDiffusion: Flexible Text-to-3D Generation with Efficient Volumetric Encoder
Zhicong Tang, Shuyang Gu, Chunyu Wang +4
This paper introduces a pioneering 3D volumetric encoder designed for text-to-3D generation. To scale up the training data for the diffusion model, a lightweight network is develop…
InstructDiffusion: A Generalist Modeling Interface for Vision Tasks
Zigang Geng, Binxin Yang, Tiankai Hang +8
We present InstructDiffusion, a unifying and generic framework for aligning computer vision tasks with human instructions. Unlike existing approaches that integrate prior knowledge…
Efficient Diffusion Training via Min-SNR Weighting Strategy
Tiankai Hang, Shuyang Gu, Chen Li +5
Denoising diffusion models have been a mainstream approach for image generation, however, training these models often suffers from slow convergence. In this paper, we discovered th…
CLIP Itself is a Strong Fine-tuner: Achieving 85.7% and 88.0% Top-1 Accuracy with ViT-B and ViT-L on ImageNet
Xiaoyi Dong, Jianmin Bao, Ting Zhang +7
Recent studies have shown that CLIP has achieved remarkable success in performing zero-shot inference while its fine-tuning performance is not satisfactory. In this paper, we ident…
Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion
Tengfei Wang, Bo Zhang, Ting Zhang +8
This paper presents a 3D generative model that uses diffusion models to automatically generate 3D digital avatars represented as neural radiance fields. A significant challenge in…
Paint by Example: Exemplar-based Image Editing with Diffusion Models
Binxin Yang, Shuyang Gu, Bo Zhang +5
Language-guided image editing has achieved great success recently. In this paper, for the first time, we investigate exemplar-guided image editing for more precise control. We achi…
Improved Vector Quantized Diffusion Models
Zhicong Tang, Shuyang Gu, Jianmin Bao +2
Vector quantized diffusion (VQ-Diffusion) is a powerful generative model for text-to-image synthesis, but sometimes can still generate low-quality samples or weakly correlated imag…
StyleSwin: Transformer-based GAN for High-resolution Image Generation
Bowen Zhang, Shuyang Gu, Bo Zhang +5
Despite the tantalizing success in a broad of vision tasks, transformers have not yet demonstrated on-par ability as ConvNets in high-resolution image generative modeling. In this…
Vector Quantized Diffusion Model for Text-to-Image Synthesis
Shuyang Gu, Dong Chen, Jianmin Bao +5
We present the vector quantized diffusion (VQ-Diffusion) model for text-to-image generation. This method is based on a vector quantized variational autoencoder (VQ-VAE) whose laten…
High-Fidelity and Arbitrary Face Editing
Yue Gao, Fangyun Wei, Jianmin Bao +4
Cycle consistency is widely used for face editing. However, we observe that the generator tends to find a tricky way to hide information from the original image to satisfy the cons…
Learnable Sampling 3D Convolution for Video Enhancement and Action Recognition
Shuyang Gu, Jianmin Bao, Dong Chen
A key challenge in video enhancement and action recognition is to fuse useful information from neighboring frames. Recent works suggest establishing accurate correspondences betwee…
PriorGAN: Real Data Prior for Generative Adversarial Nets
Shuyang Gu, Jianmin Bao, Dong Chen +1
Generative adversarial networks (GANs) have achieved rapid progress in learning rich data distributions. However, we argue about two main issues in existing techniques. First, the…
GIQA: Generated Image Quality Assessment
Shuyang Gu, Jianmin Bao, Dong Chen +1
Generative adversarial networks (GANs) have achieved impressive results today, but not all generated images are perfect. A number of quantitative criteria have recently emerged for…
Mask-Guided Portrait Editing with Conditional GANs
Shuyang Gu, Jianmin Bao, Hao Yang +3
Portrait editing is a popular subject in photo manipulation. The Generative Adversarial Network (GAN) advances the generating of realistic faces and allows more face editing. In th…
Arbitrary Style Transfer with Deep Feature Reshuffle
Shuyang Gu, Congliang Chen, Jing Liao +1
This paper introduces a novel method by reshuffling deep features (i.e., permuting the spacial locations of a feature map) of the style image for arbitrary style transfer. We theor…