85 citations · 594 across the 81 of their papers we have counts for
34 papers · 1 filter
Mitigating Artifacts in Real-World Video Super-Resolution Models
Liangbin Xie, Xintao Wang, Shuwei Shi +3
The recurrent structure is a prevalent framework for the task of video super-resolution, which models the temporal dependency between frames via hidden states. When applied to real…
Dream3D: Zero-Shot Text-to-3D Synthesis Using 3D Shape Prior and Text-to-Image Diffusion Models
Jiale Xu, Xintao Wang, Weihao Cheng +4
Recent CLIP-guided 3D optimization methods, such as DreamFields and PureCLIPNeRF, have achieved impressive results in zero-shot text-to-3D synthesis. However, due to scratch traini…
Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation
Jay Zhangjie Wu, Yixiao Ge, Xintao Wang +7
To replicate the success of text-to-image (T2I) generation, recent works employ large-scale video datasets to train a text-to-video (T2V) generator. Despite their promising results…
Rethinking the Objectives of Vector-Quantized Tokenizers for Image Synthesis
Yuchao Gu, Xintao Wang, Yixiao Ge +3
Vector-Quantized (VQ-based) generative models usually consist of two basic components, i.e., VQ tokenizers and generative transformers. Prior research focuses on improving the reco…
3D GAN Inversion with Facial Symmetry Prior
Fei Yin, Yong Zhang, Xuan Wang +8
Recently, a surge of high-quality 3D-aware GANs have been proposed, which leverage the generative power of neural rendering. It is natural to associate 3D GANs with GAN inversion m…
High-fidelity Facial Avatar Reconstruction from Monocular Video with Generative Priors
Yunpeng Bai, Yanbo Fan, Xuan Wang +4
High-fidelity facial avatar reconstruction from a monocular video is a significant research problem in computer graphics and computer vision. Recently, Neural Radiance Field (NeRF)…
Latent Video Diffusion Models for High-Fidelity Long Video Generation
Yingqing He, Tianyu Yang, Yong Zhang +2
AI-generated content has attracted lots of attention recently, but photo-realistic video synthesis is still challenging. Although many attempts using GANs and autoregressive models…
SadTalker: Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animation
Wenxuan Zhang, Xiaodong Cun, Xuan Wang +5
Generating talking head videos through a face image and a piece of speech audio still contains many challenges. ie, unnatural head movement, distorted expression, and identity modi…
Local-to-Global Registration for Bundle-Adjusting Neural Radiance Fields
Yue Chen, Xingyu Chen, Xuan Wang +4
Neural Radiance Fields (NeRF) have achieved photorealistic novel views synthesis; however, the requirement of accurate camera poses limits its application. Despite analysis-by-synt…
Darwinian Model Upgrades: Model Evolving with Selective Compatibility
Binjie Zhang, Shupeng Su, Yixiao Ge +5
The traditional model upgrading paradigm for retrieval requires recomputing all gallery embeddings before deploying the new model (dubbed as "backfilling"), which is quite expensiv…
MonoNeuralFusion: Online Monocular Neural 3D Reconstruction with Geometric Priors
Zi-Xin Zou, Shi-Sheng Huang, Yan-Pei Cao +3
High-fidelity 3D scene reconstruction from monocular videos continues to be challenging, especially for complete and fine-grained geometry reconstruction. The previous 3D reconstru…
Not All Models Are Equal: Predicting Model Transferability in a Self-challenging Fisher Space
Wenqi Shao, Xun Zhao, Yixiao Ge +5
This paper addresses an important problem of ranking the pre-trained deep neural networks and screening the most transferable ones for downstream tasks. It is challenging because t…
Music-driven Dance Regeneration with Controllable Key Pose Constraints
Junfu Pu, Ying Shan
In this paper, we propose a novel framework for music-driven dance motion synthesis with controllable key pose constraint. In contrast to methods that generate dance motion sequenc…
Learning Music-Dance Representations through Explicit-Implicit Rhythm Synchronization
Jiashuo Yu, Junfu Pu, Ying Cheng +2
Although audio-visual representation has been proved to be applicable in many downstream tasks, the representation of dancing videos, which is more specific and always accompanied…
Weakly-Supervised Temporal Action Localization by Progressive Complementary Learning
Jia-Run Du, Jia-Chang Feng, Kun-Yu Lin +5
Weakly Supervised Temporal Action Localization (WSTAL) aims to localize and classify action instances in long untrimmed videos with only video-level category labels. Due to the lac…
AnimeSR: Learning Real-World Super-Resolution Models for Animation Videos
Yanze Wu, Xintao Wang, Gen Li +1
This paper studies the problem of real-world video super-resolution (VSR) for animation videos, and reveals three key improvements for practical animation VSR. First, recent real-w…
Unleashing Vanilla Vision Transformer with Masked Image Modeling for Object Detection
Yuxin Fang, Shusheng Yang, Shijie Wang +3
We present an approach to efficiently and effectively adapt a masked image modeling (MIM) pre-trained vanilla Vision Transformer (ViT) for object detection, which is based on our t…
Accelerating the Training of Video Super-Resolution Models
Lijian Lin, Xintao Wang, Zhongang Qi +1
Despite that convolution neural networks (CNN) have recently demonstrated high-quality reconstruction for video super-resolution (VSR), efficiently training competitive VSR models…
RepSR: Training Efficient VGG-style Super-Resolution Networks with Structural Re-Parameterization and Batch Normalization
Xintao Wang, Chao Dong, Ying Shan
This paper explores training efficient VGG-style super-resolution (SR) networks with the structural re-parameterization technique. The general pipeline of re-parameterization is to…
VFHQ: A High-Quality Dataset and Benchmark for Video Face Super-Resolution
Liangbin Xie. Xintao Wang, Honglun Zhang, Chao Dong +1
Most of the existing video face super-resolution (VFSR) methods are trained and evaluated on VoxCeleb1, which is designed specifically for speaker identification and the frames in…
Do we really need temporal convolutions in action segmentation?
Dazhao Du, Bing Su, Yu Li +3
Action classification has made great progress, but segmenting and recognizing actions from long untrimmed videos remains a challenging problem. Most state-of-the-art methods focus…
Masked Image Modeling with Denoising Contrast
Kun Yi, Yixiao Ge, Xiaotong Li +5
Since the development of self-supervised visual representation learning from contrastive learning to masked image modeling (MIM), there is no significant difference in essence, tha…
VQFR: Blind Face Restoration with Vector-Quantized Dictionary and Parallel Decoder
Yuchao Gu, Xintao Wang, Liangbin Xie +4
Although generative facial prior and geometric prior have recently demonstrated high-quality results for blind face restoration, producing fine-grained facial details faithful to i…
MM-RealSR: Metric Learning based Interactive Modulation for Real-World Super-Resolution
Chong Mou, Yanze Wu, Xintao Wang +3
Interactive image restoration aims to restore images by adjusting several controlling coefficients, which determine the restoration strength. Existing methods are restricted in lea…