2 papers
cs.LG2026
SOCKET: SOft Collision Kernel EsTimator for Sparse Attention
Sahil Joshi, Agniva Chowdhury, Wyatt Bellinger +5
Exploiting sparsity during long-context inference is key to scaling large language models, as attention dominates the cost of autoregressive decoding. Sparse attention reduces this…
cs.LG2026
Sketch to Adapt: Fine-Tunable Sketches for Efficient LLM Adaptation
Tianyi Zhang, Junda Su, Aditya Desai +3
Adapting pre-trained large language models (LLMs) is crucial but challenging due to their enormous size. Parameter-efficient fine-tuning (PEFT) techniques typically employ additive…