Showing cs.LGShow all
2 papers · 1 filter
cs.LG2026
Text Capability Loss in Vision-Language Adaptation: An Attention-Sink Diagnosis
Minsik Choi, Geewook Kim, Young Geun Kim
Fine-tuning a pretrained LLM into a vision-language model (VLM) can erode the backbone's text capability, with the damage concentrated on tasks that require following exact output…
cs.LG2025
First Attentions Last: Better Exploiting First Attentions for Efficient Transformer Training
Gyudong Kim, Hyukju Na, Jin Hyeon Kim +6
As training billion-scale transformers becomes increasingly common, employing multiple distributed GPUs along with parallel training methods has become a standard practice. However…