Do All Individual Layers Help? An Empirical Study of Task-Interfering Layers in Vision-Language Models

Zhiming Liu, Yujie Wei, Lei Feng +5

Current VLMs have demonstrated capabilities across a wide range of multimodal tasks. Typically, in a pretrained VLM, all layers are engaged by default to make predictions on downst…