140 citations · 493 across the 65 of their papers we have counts for
66 papers
EditaLive! Unified Character Video Editing for Live Streaming
Zhiyuan Li, Chi-Man Pun, Peng-Tao Jiang +2
Conventional video editing primarily focuses on scene-level content, whereas live streaming places greater emphasis on the human subject. However, directly applying existing video-…
PersonaLive! Expressive Portrait Image Animation for Live Streaming
Zhiyuan Li, Chi-Man Pun, Chen Fang +2
Current diffusion-based portrait animation models predominantly focus on enhancing visual quality and expression realism, while overlooking generation latency and real-time perform…
SKEL-CF: Coarse-to-Fine Biomechanical Skeleton and Surface Mesh Recovery
Da Li, Jiping Jin, Xuanlong Yu +6
Parametric 3D human models such as SMPL have driven significant advances in human pose and shape estimation, yet their simplified kinematics limit biomechanical realism. The recent…
GenCompositor: Generative Video Compositing with Diffusion Transformer
Shuzhou Yang, Xiaoyu Li, Xiaodong Cun +4
Video compositing combines live-action footage to create video production, serving as a crucial technique in video creation and film production. Traditional pipelines require inten…
GSFixer: Improving 3D Gaussian Splatting with Reference-Guided Video Diffusion Priors
Xingyilang Yin, Qi Zhang, Jiahao Chang +6
Reconstructing 3D scenes using 3D Gaussian Splatting (3DGS) from sparse views is an ill-posed problem due to insufficient information, often resulting in noticeable artifacts. Whil…
4DVD: Cascaded Dense-view Video Diffusion Model for High-quality 4D Content Generation
Shuzhou Yang, Xiaodong Cun, Xiaoyu Li +2
Given the high complexity of directly generating high-dimensional data such as 4D, we present 4DVD, a cascaded video diffusion model that generates 4D content in a decoupled manner…
EmoCAST: Emotional Talking Portrait via Emotive Text Description
Yiguo Jiang, Xiaodong Cun, Yong Zhang +3
Emotional talking head synthesis aims to generate talking portrait videos with vivid expressions. Existing methods still exhibit limitations in control flexibility, motion naturaln…
VAU-R1: Advancing Video Anomaly Understanding via Reinforcement Fine-Tuning
Liyun Zhu, Qixiang Chen, Xi Shen +1
Video Anomaly Understanding (VAU) is essential for applications such as smart cities, security surveillance, and disaster alert systems, yet remains challenging due to its demand f…
EF-VI: Enhancing End-Frame Injection for Video Inbetweening
Liuhan Chen, Xiaodong Cun, Xiaoyu Li +5
Video inbetweening aims to synthesize intermediate video sequences conditioned on the given start and end frames. Current state-of-the-art methods primarily extend large-scale pre-…
BlobCtrl: Taming Controllable Blob for Element-level Image Editing
Yaowei Li, Lingen Li, Zhaoyang Zhang +6
As user expectations for image editing continue to rise, the demand for flexible, fine-grained manipulation of specific visual elements presents a challenge for current diffusion-b…
Mobius: Text to Seamless Looping Video Generation via Latent Shift
Xiuli Bi, Jianfei Yuan, Bo Liu +4
We present Mobius, a novel method to generate seamlessly looping videos from text descriptions directly without any user annotations, thereby creating new visual materials for the…
VideoMaker: Zero-shot Customized Video Generation with the Inherent Force of Video Diffusion Models
Tao Wu, Yong Zhang, Xiaodong Cun +6
Zero-shot customized video generation has gained significant attention due to its substantial application potential. Existing methods rely on additional models to extract and injec…
CustomTTT: Motion and Appearance Customized Video Generation via Test-Time Training
Xiuli Bi, Jian Lu, Bo Liu +4
Benefiting from large-scale pre-training of text-video pairs, current text-to-video (T2V) diffusion models can generate high-quality videos from the text description. Besides, give…
DiTCtrl: Exploring Attention Control in Multi-Modal Diffusion Transformer for Tuning-Free Multi-Prompt Longer Video Generation
Minghong Cai, Xiaodong Cun, Xiaoyu Li +5
Sora-like video generation models have achieved remarkable progress with a Multi-Modal Diffusion Transformer MM-DiT architecture. However, the current video generation models predo…
DEIM: DETR with Improved Matching for Fast Convergence
Shihua Huang, Zhichao Lu, Xiaodong Cun +3
We introduce DEIM, an innovative and efficient training framework designed to accelerate convergence in real-time object detection with Transformer-based architectures (DETR). To m…
AnchorCrafter: Animate Cyber-Anchors Selling Your Products via Human-Object Interacting Video Generation
Ziyi Xu, Ziyao Huang, Juan Cao +7
The generation of anchor-style product promotion videos presents promising opportunities in e-commerce, advertising, and consumer engagement. Despite advancements in pose-guided hu…
ForgeryTTT: Zero-Shot Image Manipulation Localization with Test-Time Training
Weihuang Liu, Xi Shen, Chi-Man Pun +1
Social media is increasingly plagued by realistic fake images, making it hard to trust content. Previous algorithms to detect these fakes often fail in new, real-world scenarios be…
Redefining Temporal Modeling in Video Diffusion: The Vectorized Timestep Approach
Yaofang Liu, Yumeng Ren, Xiaodong Cun +5
Diffusion models have revolutionized image generation, and their extension to video generation has shown promise. However, current video diffusion models~(VDMs) rely on a scalar ti…
StereoCrafter: Diffusion-based Generation of Long and High-fidelity Stereoscopic 3D from Monocular Videos
Sijie Zhao, Wenbo Hu, Xiaodong Cun +6
This paper presents a novel framework for converting 2D videos to immersive stereoscopic 3D, addressing the growing demand for 3D content in immersive experience. Leveraging founda…
DepthCrafter: Generating Consistent Long Depth Sequences for Open-world Videos
Wenbo Hu, Xiangjun Gao, Xiaoyu Li +5
Estimating video depth in open-world scenarios is challenging due to the diversity of videos in appearance, content motion, camera movement, and length. We present DepthCrafter, an…
Noise Calibration: Plug-and-play Content-Preserving Video Enhancement using Pre-trained Video Diffusion Models
Qinyu Yang, Haoxin Chen, Yong Zhang +4
In order to improve the quality of synthesized videos, currently, one predominant method involves retraining an expert diffusion model and then implementing a noising-denoising pro…
ZeroSmooth: Training-free Diffuser Adaptation for High Frame Rate Video Generation
Shaoshu Yang, Yong Zhang, Xiaodong Cun +2
Video generation has made remarkable progress in recent years, especially since the advent of the video diffusion models. Many video generation models can produce plausible synthet…
ZeroPur: Succinct Training-Free Adversarial Purification
Erhu Liu, Zonglin Yang, Bo Liu +4
Adversarial purification is a kind of defense technique that can defend against various unseen adversarial attacks without modifying the victim classifier. Existing methods often d…
CV-VAE: A Compatible Video VAE for Latent Generative Video Models
Sijie Zhao, Yong Zhang, Xiaodong Cun +5
Spatio-temporal compression of videos, utilizing networks such as Variational Autoencoders (VAE), plays a crucial role in OpenAI's SORA and numerous other video generative models.…