5 citations · 11 across the 32 of their papers we have counts for
21 papers · 1 filter
VideoCanvas: Unified Video Completion from Arbitrary Spatiotemporal Patches via In-Context Conditioning
Minghong Cai, Qiulin Wang, Zongli Ye +7
Existing controllable video generation methods are typically designed for rigid, task-specific settings, such as first-frame image-to-video, inpainting, or interpolation, treating…
UniVideo: Unified Understanding, Generation, and Editing for Videos
Cong Wei, Quande Liu, Zixuan Ye +5
Unified multimodal models have shown promising results in multimodal content generation and editing but remain largely limited to the image domain. In this work, we present UniVide…
Score Augmentation for Diffusion Models
Liang Hou, Yuan Gao, Boyuan Jiang +6
Diffusion models have achieved remarkable success in generative modeling. However, this study confirms the existence of overfitting in diffusion model training, particularly in dat…
AudioGen-Omni: A Unified Multimodal Diffusion Transformer for Video-Synchronized Audio, Speech, and Song Generation
Le Wang, Jun Wang, Chunyu Qiang +4
We present AudioGen-Omni - a unified approach based on multimodal diffusion transformers (MMDit), capable of generating high-fidelity audio, speech, and song coherently synchronize…
Imbalance in Balance: Online Concept Balancing in Generation Models
Yukai Shi, Jiarong Ou, Rui Chen +6
In visual generation tasks, the responses and combinations of complex concepts often lack stability and are error-prone, which remains an under-explored area. In this paper, we att…
Kling-Foley: Multimodal Diffusion Transformer for High-Quality Video-to-Audio Generation
Jun Wang, Xijuan Zeng, Chunyu Qiang +19
We propose Kling-Foley, a large-scale multimodal Video-to-Audio generation model that synthesizes high-quality audio synchronized with video content. In Kling-Foley, we introduce m…
FullDiT2: Efficient In-Context Conditioning for Video Diffusion Transformers
Xuanhua He, Quande Liu, Zixuan Ye +7
Fine-grained and efficient controllability on video diffusion transformers has raised increasing desires for the applicability. Recently, In-context Conditioning emerged as a power…
UNIC: Unified In-Context Video Editing
Zixuan Ye, Xuanhua He, Quande Liu +7
Recent advances in text-to-video generation have sparked interest in generative video editing tasks. Previous methods often rely on task-specific architectures (e.g., additional ad…
CamCloneMaster: Enabling Reference-based Camera Control for Video Generation
Yawen Luo, Jianhong Bai, Xiaoyu Shi +6
Camera control is crucial for generating expressive and cinematic videos. Existing methods rely on explicit sequences of camera parameters as control conditions, which can be cumbe…
Scaling Image and Video Generation via Test-Time Evolutionary Search
Haoran He, Jiajun Liang, Xintao Wang +4
As the marginal cost of scaling computation (data and parameters) during model pre-training continues to increase substantially, test-time scaling (TTS) has emerged as a promising…
A Survey of Interactive Generative Video
Jiwen Yu, Yiran Qin, Haoxuan Che +7
Interactive Generative Video (IGV) has emerged as a crucial technology in response to the growing demand for high-quality, interactive video content across various domains. In this…
VLM as Policy: Common-Law Content Moderation Framework for Short Video Platform
Xingyu Lu, Tianke Zhang, Chang Meng +14
Exponentially growing short video platforms (SVPs) face significant challenges in moderating content detrimental to users' mental health, particularly for minors. The dissemination…
FullDiT: Multi-Task Video Generative Foundation Model with Full Attention
Xuan Ju, Weicai Ye, Quande Liu +6
Current video generative foundation models primarily focus on text-to-video tasks, providing limited control for fine-grained video content creation. Although adapter-based approac…
DiffMoE: Dynamic Token Selection for Scalable Diffusion Transformers
Minglei Shi, Ziyang Yuan, Haotian Yang +10
Diffusion models have demonstrated remarkable success in various image generation tasks, but their performance is often limited by the uniform processing of inputs across varying c…
Any2Caption:Interpreting Any Condition to Caption for Controllable Video Generation
Shengqiong Wu, Weicai Ye, Jiahao Wang +8
To address the bottleneck of accurate user intent interpretation within the current video generation community, we present Any2Caption, a novel framework for controllable video gen…
Boosting Resolution Generalization of Diffusion Transformers with Randomized Positional Encodings
Liang Hou, Cong Liu, Mingwu Zheng +4
Resolution generalization in image generation tasks enables the production of higher-resolution images with lower training resolution overhead. However, a key obstacle for diffusio…
Finedeep: Mitigating Sparse Activation in Dense LLMs via Multi-Layer Fine-Grained Experts
Leiyu Pan, Zhenpeng Su, Minxuan Lv +10
Large language models have demonstrated exceptional performance across a wide range of tasks. However, dense models usually suffer from sparse activation, where many activation val…
CineMaster: A 3D-Aware and Controllable Framework for Cinematic Text-to-Video Generation
Qinghe Wang, Yawen Luo, Xiaoyu Shi +7
In this work, we present CineMaster, a novel framework for 3D-aware and controllable text-to-video generation. Our goal is to empower users with comparable controllability as profe…
DSMoE: Matrix-Partitioned Experts with Dynamic Routing for Computation-Efficient Dense LLMs
Minxuan Lv, Zhenpeng Su, Leiyu Pan +10
As large language models continue to scale, computational costs and resource consumption have emerged as significant challenges. While existing sparsification methods like pruning…
Improving Video Generation with Human Feedback
Jie Liu, Gongye Liu, Jiajun Liang +14
Video generation has achieved significant advances through rectified flow techniques, but issues like unsmooth motion and misalignment between videos and prompts persist. In this w…
ConceptMaster: Multi-Concept Video Customization on Diffusion Transformer Models Without Test-Time Tuning
Yuzhou Huang, Ziyang Yuan, Quande Liu +6
Text-to-video generation has made remarkable advancements through diffusion models. However, Multi-Concept Video Customization (MCVC) remains a significant challenge. We identify t…