Showing cs.LGShow all
2 papers · 1 filter
cs.LG2026
Predicting and Repairing Merge Collapse in Large Language Models
Jungseob Lee, Seungyoon Lee, Sugyeong Eo +3
Large language models fine-tuned from a shared base can be merged by averaging their task vectors, but some merges collapse far below the base model, and common merge operators giv…
cs.LG2026
Where Activation Sparsity and KV-Cache Sparsity Cross in LLM Decoding
Jungseob Lee, Seungyoon Lee, Seongtae Hong +2
At each step, decoding one sequence with a large language model rereads the projection weights, whose traffic is fixed, and the key-value (KV) cache, whose traffic grows with conte…