6 papers
Addressing the ID-Matching Challenge in Long Video Captioning
Zhantao Yang, Huangji Wang, Ruili Feng +6
Generating captions for long and complex videos is both critical and challenging, with significant implications for the growing fields of text-to-video generation and multi-modal u…
MAMBO-G: Magnitude-Aware Mitigation for Boosted Guidance
Shangwen Zhu, Qianyu Peng, Zhilei Shu +9
High-fidelity text-to-image and text-to-video generation typically relies on Classifier-Free Guidance (CFG), but achieving optimal results often demands computationally expensive s…
Instability in Diffusion ODEs: An Explanation for Inaccurate Image Reconstruction
Han Zhang, Jinghong Mao, Shangwen Zhu +6
Diffusion reconstruction plays a critical role in various applications such as image editing, restoration, and style transfer. In theory, the reconstruction should be simple - it j…
Accelerating Diffusion Sampling via Exploiting Local Transition Coherence
Shangwen Zhu, Han Zhang, Zhantao Yang +4
Text-based diffusion models have made significant breakthroughs in generating high-quality images and videos from textual descriptions. However, the lengthy sampling time of the de…
The Matrix: Infinite-Horizon World Generation with Real-Time Moving Control
Ruili Feng, Han Zhang, Zhantao Yang +7
We present The Matrix, the first foundational realistic world simulator capable of generating continuous 720p high-fidelity real-scene video streams with real-time, responsive cont…
BACON: Improving Clarity of Image Captions via Bag-of-Concept Graphs
Zhantao Yang, Ruili Feng, Keyu Yan +13
Advancements in large Vision-Language Models have brought precise, accurate image captioning, vital for advancing multi-modal image understanding and processing. Yet these captions…