Visual Instruction Tuning Aligns Modalities through Abstraction

Luis Palacios, Lorenzo Basile, Diego Doimo +1

Visual instruction tuning effectively adapts a pre-trained Large Language Model (LLM) to process image information alongside text. Yet, it remains unclear how visual features are e…