Llama-Mobile: Efficient 2.7-Bit Quantization of VLMs

Luka Ribar, Jeevan Bhoot, Douglas Orr

Deploying vision-language models (VLMs) on mobile devices is challenging due to their significant memory and compute requirements. We present a framework for quantizing VLMs for ef…