Fitting Vision Adapters at Frontier Scales

Jaehoon Lee, Harry Partridge, Mudith Jayasekara +3

Training a small projector between a frozen vision encoder and language model is an established approach to multimodal learning. As the parameter count of language models scales dr…