6 papers
Imagine Before You Draw: Visual Prompt Engineering for Image Generation
Liyu Jia, Fengda Zhang, Jiachun Pan +7
Incorporating visual semantic representations as an intermediate step before image generation can reduce the modeling difficulty between text and images, thereby improving generati…
WEAVE: Unleashing and Benchmarking the In-context Interleaved Comprehension and Generation
Wei Chow, Jiachun Pan, Yongyuan Liang +10
Recent advances in unified multimodal models (UMMs) have enabled impressive progress in visual comprehension and generation. However, existing datasets and benchmarks focus primari…
Selftok: Discrete Visual Tokens of Autoregression, by Diffusion, and for Reasoning
Bohan Wang, Zhongqi Yue, Fengda Zhang +15
We completely discard the conventional spatial prior in image representation and introduce a novel discrete visual tokenizer: Self-consistency Tokenizer (Selftok). At its design co…
On Path to Multimodal Generalist: General-Level and General-Bench
Hao Fei, Yuan Zhou, Juncheng Li +29
The Multimodal Large Language Model (MLLM) is currently experiencing rapid growth, driven by the advanced capabilities of LLMs. Unlike earlier specialists, existing MLLMs are evolv…
Reasoning Physical Video Generation with Diffusion Timestep Tokens via Reinforcement Learning
Wang Lin, Liyu Jia, Wentao Hu +6
Despite recent progress in video generation, producing videos that adhere to physical laws remains a significant challenge. Traditional diffusion-based methods struggle to extrapol…
Generative Multimodal Pretraining with Discrete Diffusion Timestep Tokens
Kaihang Pan, Wang Lin, Zhongqi Yue +6
Recent endeavors in Multimodal Large Language Models (MLLMs) aim to unify visual comprehension and generation by combining LLM and diffusion models, the state-of-the-art in each ta…