11 papers · 1 filter
Scaling Unified Multimodal Models to Long-Horizon Interleaved Generation
Ziqiao Peng, Yutao Cui, Wang Lin +12
Unified multimodal models (UMMs) provide an interface for interleaved text-image generation, where generated visual states can directly inform subsequent reasoning and synthesis. H…
WithEveryone: Unified Planning and Identity Grounding for Group Image Generation
Hengyuan Xu, Qixun Wang, Yiji Cheng +5
Identity-preserving image generation becomes increasingly unreliable when a scene must contain many specified people. Beyond retaining each identity, the model must bind every refe…
Twins: Learn to Predict Unified Representations with Focal Loss
Kaixiong Gong, Xin Cai, Bin Lin +9
Unified multimodal models seek a shared visual token space that supports both multimodal understanding and image generation. Discrete methods unify the interface via a shared codeb…
Think, Plan, Paint: Layout-Aware Reasoning for Controllable Image Generation in Unified Models
Junhao Liu, Jian-Wei Zhang, Tao Huang +3
Unified Multimodal Large Language Models (MLLMs) offer a promising paradigm for unifying visual understanding and generation, yet they still struggle to follow complex spatial inst…
Rosetta: Composable Native Multimodal Pretraining
Xiangyue Liu, Zijian Zhang, Miles Yang +3
Achieving true artificial general intelligence requires foundation models capable of integrating new modalities without forgetting prior knowledge. However, accommodating continuou…
GEAR: Guided End-to-End AutoRegression for Image Synthesis
Bin Lin, Zheyuan Liu, Chenguo Lin +8
Visual generative models are typically trained in two stages. A tokenizer is first trained for reconstruction and then frozen, after which a generator is trained on its discrete in…
HYDRA-X: Native Unified Multimodal Models with Holistic Visual Tokenizers
Guozhen Zhang, Xuerui Qiu, Yutao Cui +11
Holistic visual tokenizers are fundamental to unified multimodal models (UMMs) as they map diverse visual inputs into a unified representation space. In this paper, we present HYDR…
Symbiotic-MoE: Unlocking the Synergy between Generation and Understanding
Xiangyue Liu, Zijian Zhang, Miles Yang +3
Empowering Large Multimodal Models (LMMs) with image generation often leads to catastrophic forgetting in understanding tasks due to severe gradient conflicts. While existing parad…
HYDRA: Representation Harmonized Tokenization for Multimodal Generation and Understanding
Xuerui Qiu, Yutao Cui, Guozhen Zhang +10
Unified Multimodal Models struggle to bridge the fundamental gap between the abstract representations needed for visual understanding and the detailed primitives required for gener…
UniCom: Unified Multimodal Modeling via Compressed Continuous Semantic Representations
Yaqi Zhao, Wang Lin, Zijian Zhang +5
Current unified multimodal models typically rely on discrete visual tokenizers to bridge the modality gap. However, discretization inevitably discards fine-grained semantic informa…
iFSQ: Improving FSQ for Image Generation with 1 Line of Code
Bin Lin, Zongjian Li, Yuwei Niu +9
The field of image generation is currently bifurcated into autoregressive (AR) models operating on discrete tokens and diffusion models utilizing continuous latents. This divide, r…