3 citations · 7 across the 20 of their papers we have counts for
23 papers · 1 filter
PixelMoE: Scaling Pixel-Space Diffusion with LLM-Style Mixture-of-Experts
Ruixiao Dong, Jianmin Bao, Bozhi Luan +6
Pixel-space diffusion models provide an appealing alternative to latent-space diffusion models by directly modeling raw images without relying on a learned tokenizer. However, scal…
Distributed Implicit Harm: A Compositional Safety Blind Spot in MLLM-Based Video Moderation
Ruotong Wang, Zihao Zhu, Siwei Lyu +2
Despite their growing use in video moderation, multimodal large language models (MLLMs) exhibit a compositional safety blind spot: videos composed of seemingly benign components ca…
SRC-Flow: Compact Semantic Representations Enable Normalizing Flows for Image Generation
Longtao Jiang, Jianmin Bao, Zhendong Wang +4
Normalizing flows (NFs) provide exact likelihoods and deterministic invertible sampling, but have historically lagged behind diffusion models for large-scale image generation. We i…
SALAD: Achieve High-Sparsity Attention via Efficient Linear Attention Tuning for Video Diffusion Transformer
Tongcheng Fang, Hanling Zhang, Ruiqi Xie +8
Diffusion Transformers have demonstrated remarkable performance in video generation. However, their long input sequences incur substantial latency due to the quadratic complexity o…
Imbalance in Balance: Online Concept Balancing in Generation Models
Yukai Shi, Jiarong Ou, Rui Chen +6
In visual generation tasks, the responses and combinations of complex concepts often lack stability and are error-prone, which remains an under-explored area. In this paper, we att…
VMoBA: Mixture-of-Block Attention for Video Diffusion Models
Jianzong Wu, Liang Hou, Haotian Yang +5
The quadratic complexity of full attention mechanisms poses a significant bottleneck for Video Diffusion Models (VDMs) aiming to generate long-duration, high-resolution videos. Whi…
Training-Free Efficient Video Generation via Dynamic Token Carving
Yuechen Zhang, Jinbo Xing, Bin Xia +6
Despite the remarkable generation quality of video Diffusion Transformer (DiT) models, their practical deployment is severely hindered by extensive computational requirements. This…
VFRTok: Variable Frame Rates Video Tokenizer with Duration-Proportional Information Assumption
Tianxiong Zhong, Xingye Tian, Boyuan Jiang +4
Modern video generation frameworks based on Latent Diffusion Models suffer from inefficiencies in tokenization due to the Frame-Proportional Information Assumption. Existing tokeni…
BadVideo: Stealthy Backdoor Attack against Text-to-Video Generation
Ruotong Wang, Mingli Zhu, Jiarong Ou +4
Text-to-video (T2V) generative models have rapidly advanced and found widespread applications across fields like entertainment, education, and marketing. However, the adversarial v…
DiffMoE: Dynamic Token Selection for Scalable Diffusion Transformers
Minglei Shi, Ziyang Yuan, Haotian Yang +10
Diffusion models have demonstrated remarkable success in various image generation tasks, but their performance is often limited by the uniform processing of inputs across varying c…
MTV-Inpaint: Multi-Task Long Video Inpainting
Shiyuan Yang, Zheng Gu, Liang Hou +4
Video inpainting involves modifying local regions within a video, ensuring spatial and temporal consistency. Most existing methods focus primarily on scene completion (i.e., fillin…
Boosting Resolution Generalization of Diffusion Transformers with Randomized Positional Encodings
Liang Hou, Cong Liu, Mingwu Zheng +4
Resolution generalization in image generation tasks enables the production of higher-resolution images with lower training resolution overhead. However, a key obstacle for diffusio…
RectifiedHR: Enable Efficient High-Resolution Synthesis via Energy Rectification
Zhen Yang, Guibao Shen, Minyang Li +5
Diffusion models have achieved remarkable progress across various visual generation tasks. However, their performance significantly declines when generating content at resolutions…
Towards Precise Scaling Laws for Video Diffusion Transformers
Yuanyang Yin, Yaqi Zhao, Mingwu Zheng +11
Achieving optimal performance of video diffusion transformers within given data and compute budget is crucial due to their high training costs. This necessitates precisely determin…
Owl-1: Omni World Model for Consistent Long Video Generation
Yuanhui Huang, Wenzhao Zheng, Yuan Gao +5
Video generation models (VGMs) have received extensive attention recently and serve as promising candidates for general-purpose large vision models. While they can only generate sh…
Koala-36M: A Large-scale Video Dataset Improving Consistency between Fine-grained Conditions and Video Content
Qiuheng Wang, Yukai Shi, Jiarong Ou +10
With the continuous progress of visual generation technologies, the scale of video datasets has grown exponentially. The quality of these datasets plays a pivotal role in the perfo…
SEA: Supervised Embedding Alignment for Token-Level Visual-Textual Integration in MLLMs
Yuanyang Yin, Yaqi Zhao, Yajie Zhang +7
Multimodal Large Language Models (MLLMs) have demonstrated remarkable capabilities by integrating visual and textual inputs, yet modality alignment remains one of the most challeng…
VideoTetris: Towards Compositional Text-to-Video Generation
Ye Tian, Ling Yang, Haotian Yang +9
Diffusion models have demonstrated great success in text-to-video (T2V) generation. However, existing methods may face challenges when handling complex (long) video generation scen…
SG-Adapter: Enhancing Text-to-Image Generation with Scene Graph Guidance
Guibao Shen, Luozhou Wang, Jiantao Lin +9
Recent advancements in text-to-image generation have been propelled by the development of diffusion models and multi-modality learning. However, since text is typically represented…
UNIAA: A Unified Multi-modal Image Aesthetic Assessment Baseline and Benchmark
Zhaokun Zhou, Qiulin Wang, Bin Lin +7
As an alternative to expensive expert evaluation, Image Aesthetic Assessment (IAA) stands out as a crucial task in computer vision. However, traditional IAA methods are typically c…
Motion Inversion for Video Customization
Luozhou Wang, Ziyang Mai, Guibao Shen +6
In this work, we present a novel approach for motion customization in video generation, addressing the widespread gap in the exploration of motion representation within video gener…
DVIS++: Improved Decoupled Framework for Universal Video Segmentation
Tao Zhang, Xingye Tian, Yikang Zhou +7
We present the \textbf{D}ecoupled \textbf{VI}deo \textbf{S}egmentation (DVIS) framework, a novel approach for the challenging task of universal video segmentation, including video…
Stable Segment Anything Model
Qi Fan, Xin Tao, Lei Ke +6
The Segment Anything Model (SAM) achieves remarkable promptable segmentation given high-quality prompts which, however, often require good skills to specify. To make SAM robust to…