HEED: Density-Weighted Residual Alignment for Hybrid Vision-Language Model Distillation

Yihao Liang, Niraj K. Jha

Distilling vision-language models into faster hybrid architectures, such as 3:1 Mamba-2/attention mixes, is now standard practice for making inference efficient. Aggregate benchmar…