From Generalist to Specialist: Adapting Vision Language Models via Task-Specific Visual Instruction Tuning

Yang Bai, Yang Zhou, Jun Zhou +3

Large vision language models (VLMs) combine large language models with vision encoders, demonstrating promise across various tasks. However, they often underperform in task-specifi…