37 citations · 85 across the 19 of their papers we have counts for
19 papers
VideoMaker: Zero-shot Customized Video Generation with the Inherent Force of Video Diffusion Models
Tao Wu, Yong Zhang, Xiaodong Cun +6
Zero-shot customized video generation has gained significant attention due to its substantial application potential. Existing methods rely on additional models to extract and injec…
Consistent Human Image and Video Generation with Spatially Conditioned Diffusion
Mingdeng Cao, Chong Mou, Ziyang Yuan +4
Consistent human-centric image and video synthesis aims to generate images or videos with new poses while preserving appearance consistency with a given reference image, which is c…
Noise Calibration: Plug-and-play Content-Preserving Video Enhancement using Pre-trained Video Diffusion Models
Qinyu Yang, Haoxin Chen, Yong Zhang +4
In order to improve the quality of synthesized videos, currently, one predominant method involves retraining an expert diffusion model and then implementing a noising-denoising pro…
MiraData: A Large-Scale Video Dataset with Long Durations and Structured Captions
Xuan Ju, Yiming Gao, Zhaoyang Zhang +6
Sora's high-motion intensity and long consistent videos have significantly impacted the field of video generation, attracting unprecedented attention. However, existing publicly av…
ZeroSmooth: Training-free Diffuser Adaptation for High Frame Rate Video Generation
Shaoshu Yang, Yong Zhang, Xiaodong Cun +2
Video generation has made remarkable progress in recent years, especially since the advent of the video diffusion models. Many video generation models can produce plausible synthet…
CV-VAE: A Compatible Video VAE for Latent Generative Video Models
Sijie Zhao, Yong Zhang, Xiaodong Cun +5
Spatio-temporal compression of videos, utilizing networks such as Variational Autoencoders (VAE), plays a crucial role in OpenAI's SORA and numerous other video generative models.…
MOFA-Video: Controllable Image Animation via Generative Motion Field Adaptions in Frozen Image-to-Video Diffusion Model
Muyao Niu, Xiaodong Cun, Xintao Wang +3
We present MOFA-Video, an advanced controllable image animation method that generates video from the given image using various additional controllable signals (such as human landma…
BrushNet: A Plug-and-Play Image Inpainting Model with Decomposed Dual-Branch Diffusion
Xuan Ju, Xian Liu, Xintao Wang +3
Image inpainting, the process of restoring corrupted images, has seen significant advancements with the advent of diffusion models (DMs). Despite these advancements, current DM ada…
Make a Cheap Scaling: A Self-Cascade Diffusion Model for Higher-Resolution Adaptation
Lanqing Guo, Yingqing He, Haoxin Chen +9
Diffusion models have proven to be highly effective in image and video generation; however, they encounter challenges in the correct composition of objects when generating images o…
VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Models
Haoxin Chen, Yong Zhang, Xiaodong Cun +4
Text-to-video generation aims to produce a video based on a given prompt. Recently, several commercial video models have been able to generate plausible videos with minimal noise,…
Towards A Better Metric for Text-to-Video Generation
Jay Zhangjie Wu, Guian Fang, Haoning Wu +11
Generative models have demonstrated remarkable capability in synthesizing high-quality text, images, and videos. For video generation, contemporary text-to-video models exhibit imp…
AnimateZero: Video Diffusion Models are Zero-Shot Image Animators
Jiwen Yu, Xiaodong Cun, Chenyang Qi +4
Large-scale text-to-video (T2V) diffusion models have great progress in recent years in terms of visual quality, motion and temporal consistency. However, the generation process is…
MagicStick: Controllable Video Editing via Control Handle Transformations
Yue Ma, Xiaodong Cun, Sen Liang +5
Text-based video editing has recently attracted considerable interest in changing the style or replacing the objects with a similar structure. Beyond this, we demonstrate that prop…
StyleCrafter: Enhancing Stylized Text-to-Video Generation with Style Adapter
Gongye Liu, Menghan Xia, Yong Zhang +6
Text-to-video (T2V) models have shown remarkable capabilities in generating diverse videos. However, they struggle to produce user-desired stylized videos due to (i) text's inheren…
DynamiCrafter: Animating Open-domain Images with Video Diffusion Priors
Jinbo Xing, Menghan Xia, Yong Zhang +6
Animating a still image offers an engaging visual experience. Traditional image animation techniques mainly focus on animating natural scenes with stochastic dynamics (e.g. clouds…
VideoCrafter1: Open Diffusion Models for High-Quality Video Generation
Haoxin Chen, Menghan Xia, Yingqing He +9
Video generation has increasingly gained interest in both academia and industry. Although commercial tools can generate plausible videos, there is a limited number of open-source m…
ScaleCrafter: Tuning-free Higher-Resolution Visual Generation with Diffusion Models
Yingqing He, Shaoshu Yang, Haoxin Chen +7
In this work, we investigate the capability of generating images from pre-trained diffusion models at much higher resolutions than the training image sizes. In addition, the genera…
Making LLaMA SEE and Draw with SEED Tokenizer
Yuying Ge, Sijie Zhao, Ziyun Zeng +4
The great success of Large Language Models (LLMs) has expanded the potential of multimodality, contributing to the gradual evolution of General Artificial Intelligence (AGI). A tru…
EvalCrafter: Benchmarking and Evaluating Large Video Generation Models
Yaofang Liu, Xiaodong Cun, Xuebo Liu +7
The vision and language generative models have been overgrown in recent years. For video generation, various open-sourced models and public-available services have been developed t…