10 papers
StreamDiT: Real-Time Streaming Text-to-Video Generation
Akio Kodaira, Tingbo Hou, Ji Hou +4
Recently, great progress has been achieved in text-to-video (T2V) generation by scaling transformer-based diffusion models to billions of parameters, which can generate high-qualit…
Non-Markov Multi-Round Conversational Image Generation with History-Conditioned MLLMs
Haochen Zhang, Animesh Sinha, Felix Juefei-Xu +8
Conversational image generation requires a model to follow user instructions across multiple rounds of interaction, grounded in interleaved text and images that accumulate as chat…
Llama Learns to Direct: DirectorLLM for Human-Centric Video Generation
Kunpeng Song, Tingbo Hou, Zecheng He +12
In this paper, we introduce DirectorLLM, a novel video generation model that employs a large language model (LLM) to orchestrate human poses within videos. As foundational text-to-…
Populate-A-Scene: Affordance-Aware Human Video Generation
Mengyi Shan, Zecheng He, Haoyu Ma +4
Can a video generation model be repurposed as an interactive world simulator? We explore the affordance perception potential of text-to-video models by teaching them to predict hum…
LinGen: Towards High-Resolution Minute-Length Text-to-Video Generation with Linear Computational Complexity
Hongjie Wang, Chih-Yao Ma, Yen-Cheng Liu +10
Text-to-video generation enhances content creation but is highly computationally intensive: The computational cost of Diffusion Transformers (DiTs) scales quadratically in the numb…
Token-Shuffle: Towards High-Resolution Image Generation with Autoregressive Models
Xu Ma, Peize Sun, Haoyu Ma +22
Autoregressive (AR) models, long dominant in language generation, are increasingly applied to image synthesis but are often considered less competitive than Diffusion-based models.…