8 papers
Amortized Moment Matching for Visual Generation
Wenze Liu, Xintao Wang, Pengfei Wan +1
We propose amortized moment matching, utilizing neural networks to learn data moments as distributional training signals. By casting diffusion denoisers through polynomial projecti…
In-Context Audio Control of Video Diffusion Transformers
Wenze Liu, Weicai Ye, Minghong Cai +3
Recent advancements in video generation have seen a shift towards unified, transformer-based foundation models that can handle multiple conditional inputs in-context. However, thes…
VideoCanvas: Unified Video Completion from Arbitrary Spatiotemporal Patches via In-Context Conditioning
Minghong Cai, Qiulin Wang, Zongli Ye +7
Existing controllable video generation methods are typically designed for rigid, task-specific settings, such as first-frame image-to-video, inpainting, or interpolation, treating…
Learning to Integrate Diffusion ODEs by Averaging the Derivatives
Wenze Liu, Xiangyu Yue
To accelerate diffusion model inference, numerical solvers perform poorly at extremely small steps, while distillation techniques often introduce complexity and instability. This w…
DiTCtrl: Exploring Attention Control in Multi-Modal Diffusion Transformer for Tuning-Free Multi-Prompt Longer Video Generation
Minghong Cai, Xiaodong Cun, Xiaoyu Li +5
Sora-like video generation models have achieved remarkable progress with a Multi-Modal Diffusion Transformer MM-DiT architecture. However, the current video generation models predo…
Customize Your Visual Autoregressive Recipe with Set Autoregressive Modeling
Wenze Liu, Le Zhuo, Yi Xin +3
We introduce a new paradigm for AutoRegressive (AR) image generation, termed Set AutoRegressive Modeling (SAR). SAR generalizes the conventional AR to the next-set setting, i.e., s…
Training Matting Models without Alpha Labels
Wenze Liu, Zixuan Ye, Hao Lu +2
The labelling difficulty has been a longstanding problem in deep image matting. To escape from fine labels, this work explores using rough annotations such as trimaps coarsely indi…
Lumina-Next: Making Lumina-T2X Stronger and Faster with Next-DiT
Le Zhuo, Ruoyi Du, Han Xiao +19
Lumina-T2X is a nascent family of Flow-based Large Diffusion Transformers that establishes a unified framework for transforming noise into various modalities, such as images and vi…