1.2k citations · 1.8k across the 39 of their papers we have counts for
40 papers
DUO-VSR: Dual-Stream Distillation for One-Step Video Super-Resolution
Zhengyao Lv, Menghan Xia, Xintao Wang +1
Diffusion-based video super-resolution (VSR) has recently achieved remarkable fidelity but still suffers from prohibitive sampling costs. While distribution matching distillation (…
Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wild
Fanghua Yu, Jinjin Gu, Zheyuan Li +6
We introduce SUPIR (Scaling-UP Image Restoration), a groundbreaking image restoration method that harnesses generative prior and the power of model scaling up. Leveraging multi-mod…
Unifying Image Processing as Visual Prompting Question Answering
Yihao Liu, Xiangyu Chen, Xianzheng Ma +4
Image processing is a fundamental task in computer vision, which aims at enhancing image quality and extracting essential features for subsequent vision applications. Traditionally…
HAT: Hybrid Attention Transformer for Image Restoration
Xiangyu Chen, Xintao Wang, Wenlong Zhang +4
Transformer-based methods have shown impressive performance in image restoration tasks, such as image super-resolution and denoising. However, we find that these networks can only…
Planting a SEED of Vision in Large Language Model
Yuying Ge, Yixiao Ge, Ziyun Zeng +2
We present SEED, an elaborate image tokenizer that empowers Large Language Models (LLMs) with the emergent ability to SEE and Draw at the same time. Research on image tokenizers ha…
GET3D--: Learning GET3D from Unconstrained Image Collections
Fanghua Yu, Xintao Wang, Zheyuan Li +3
The demand for efficient 3D model generation techniques has grown exponentially, as manual creation of 3D models is time-consuming and requires specialized expertise. While generat…
Animate-A-Story: Storytelling with Retrieval-Augmented Video Generation
Yingqing He, Menghan Xia, Haoxin Chen +8
Generating videos for visual storytelling can be a tedious and complex process that typically requires either live-action filming or graphics animation rendering. To bypass these c…
DeSRA: Detect and Delete the Artifacts of GAN-based Real-World Super-Resolution Models
Liangbin Xie, Xintao Wang, Xiangyu Chen +4
Image super-resolution (SR) with generative adversarial networks (GAN) has achieved great success in restoring realistic details. However, it is notorious that GAN-based SR models…
DragonDiffusion: Enabling Drag-style Manipulation on Diffusion Models
Chong Mou, Xintao Wang, Jiechong Song +2
Despite the ability of existing large-scale text-to-image (T2I) models to generate high-quality images from detailed textual descriptions, they often lack the ability to precisely…
DreamDiffusion: Generating High-Quality Images from Brain EEG Signals
Yunpeng Bai, Xintao Wang, Yan-pei Cao +3
This paper introduces DreamDiffusion, a novel method for generating high-quality images directly from brain electroencephalogram (EEG) signals, without the need to translate though…
InstructP2P: Learning to Edit 3D Point Clouds with Text Instructions
Jiale Xu, Xintao Wang, Yan-Pei Cao +3
Enhancing AI systems to perform tasks following human instructions can significantly boost productivity. In this paper, we present InstructP2P, an end-to-end framework for 3D shape…
Make-Your-Video: Customized Video Generation Using Textual and Structural Guidance
Jinbo Xing, Menghan Xia, Yuxin Liu +9
Creating a vivid video from the event or scenario in our imagination is a truly fascinating experience. Recent advancements in text-to-video synthesis have unveiled the potential t…
Inserting Anybody in Diffusion Models via Celeb Basis
Ge Yuan, Xiaodong Cun, Yong Zhang +5
Exquisite demand exists for customizing the pretrained large text-to-image model, , Stable Diffusion, to generate innovative concepts, such as the users themselves.…
TaleCrafter: Interactive Story Visualization with Multiple Characters
Yuan Gong, Youxin Pang, Xiaodong Cun +8
Accurate Story visualization requires several necessary elements, such as identity consistency across frames, the alignment between plain text and visual content, and a reasonable…
Mix-of-Show: Decentralized Low-Rank Adaptation for Multi-Concept Customization of Diffusion Models
Yuchao Gu, Xintao Wang, Jay Zhangjie Wu +10
Public large-scale text-to-image diffusion models, such as Stable Diffusion, have gained significant attention from the community. These models can be easily customized for new con…
MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editing
Mingdeng Cao, Xintao Wang, Zhongang Qi +3
Despite the success in large-scale text-to-image generation and text-conditioned image editing, existing methods still struggle to produce consistent generation and editing results…
Follow Your Pose: Pose-Guided Text-to-Video Generation using Pose-Free Videos
Yue Ma, Yingqing He, Xiaodong Cun +5
Generating text-editable and pose-controllable character videos have an imperious demand in creating various digital human. Nevertheless, this task has been restricted by the absen…
T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models
Chong Mou, Xintao Wang, Liangbin Xie +5
The incredible generative ability of large-scale text-to-image (T2I) models has demonstrated strong power of learning complex structures and meaningful semantics. However, relying…
FateZero: Fusing Attentions for Zero-shot Text-based Video Editing
Chenyang Qi, Xiaodong Cun, Yong Zhang +4
The diffusion-based generative models have achieved remarkable success in text-based image generation. However, since it contains enormous randomness in generation progress, it is…
OSRT: Omnidirectional Image Super-Resolution with Distortion-aware Transformer
Fanghua Yu, Xintao Wang, Mingdeng Cao +3
Omnidirectional images (ODIs) have obtained lots of research interest for immersive experiences. Although ODIs require extremely high resolution to capture details of the entire sc…
Mitigating Artifacts in Real-World Video Super-Resolution Models
Liangbin Xie, Xintao Wang, Shuwei Shi +3
The recurrent structure is a prevalent framework for the task of video super-resolution, which models the temporal dependency between frames via hidden states. When applied to real…
Dream3D: Zero-Shot Text-to-3D Synthesis Using 3D Shape Prior and Text-to-Image Diffusion Models
Jiale Xu, Xintao Wang, Weihao Cheng +4
Recent CLIP-guided 3D optimization methods, such as DreamFields and PureCLIPNeRF, have achieved impressive results in zero-shot text-to-3D synthesis. However, due to scratch traini…
Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation
Jay Zhangjie Wu, Yixiao Ge, Xintao Wang +7
To replicate the success of text-to-image (T2I) generation, recent works employ large-scale video datasets to train a text-to-video (T2V) generator. Despite their promising results…
Reference-based Image and Video Super-Resolution via C2-Matching
Yuming Jiang, Kelvin C. K. Chan, Xintao Wang +2
Reference-based Super-Resolution (Ref-SR) has recently emerged as a promising paradigm to enhance a low-resolution (LR) input image or video by introducing an additional high-resol…