Understanding Unified VLMs: From Attention Probing to Guidance Tuning for Multimodal Generation

Zhengyang Luo, Xiaoyu Qiu, Zhiheng Tan +3

Unified vision-language models (VLMs) perform both text-to-image (T2I) and image-to-text (I2T) generation with one shared Transformer backbone, yet how the backbone processes the c…