Prioritizing Image-Related Tokens Enhances Vision-Language Pre-Training

Yangyi Chen, Hao Peng, Tong Zhang +1

In standard large vision-language models (LVLMs) pre-training, the model typically maximizes the joint probability of the caption conditioned on the image via next-token prediction…