8 papers
QK-Edit: Revisiting Attention-based Injection in MM-DiT for Image and Video Editing
Tiancheng Shen, Zilong Huang, Xiangtai Li +6
Multimodal Diffusion Transformers (MM-DiTs) have recently emerged as a powerful framework for unified text-vision synthesis, surpassing traditional U-Net architectures in generativ…
DreamShot: Personalized Storyboard Synthesis with Video Diffusion Prior
Junjia Huang, Binbin Yang, Pengxiang Yan +6
Storyboard synthesis plays a crucial role in visual storytelling, aiming to generate coherent shot sequences that visually narrate cinematic events with consistent characters, scen…
NextFlow: Unified Sequential Modeling Activates Multimodal Understanding and Generation
Huichao Zhang, Liao Qu, Yiheng Liu +33
We present NextFlow, a unified decoder-only autoregressive transformer trained on 6 trillion interleaved text-image discrete tokens. By leveraging a unified vision representation w…
DreamOmni3: Scribble-based Editing and Generation
Bin Xia, Bohao Peng, Jiyang Liu +8
Recently unified generation and editing models have achieved remarkable success with their impressive performance. These models rely mainly on text prompts for instruction-based ed…
DreamOmni2: Multimodal Instruction-based Editing and Generation
Bin Xia, Bohao Peng, Yuechen Zhang +10
Recent advancements in instruction-based image editing and subject-driven generation have garnered significant attention, yet both tasks still face limitations in meeting practical…
DreamVE: Unified Instruction-based Image and Video Editing
Bin Xia, Jiyang Liu, Yuechen Zhang +6
Instruction-based editing holds vast potential due to its simple and efficient interactive editing format. However, instruction-based editing, particularly for video, has been cons…
DreamFuse: Adaptive Image Fusion with Diffusion Transformer
Junjia Huang, Pengxiang Yan, Jiyang Liu +5
Image fusion seeks to seamlessly integrate foreground objects with background scenes, producing realistic and harmonious fused images. Unlike existing methods that directly insert…
DreamLayer: Simultaneous Multi-Layer Generation via Diffusion Mode
Junjia Huang, Pengxiang Yan, Jinhang Cai +5
Text-driven image generation using diffusion models has recently gained significant attention. To enable more flexible image manipulation and editing, recent research has expanded…