34 citations · 82 across the 8 of their papers we have counts for
3 papers · 1 filter
Sparse Upcycling: Inference Inefficient Finetuning
Sasha Doubov, Nikhil Sardana, Vitaliy Chiley
Small, highly trained, open-source large language models are widely used due to their inference efficiency, but further improving their quality remains a challenge. Sparse upcyclin…
Inference-Friendly Models With MixAttention
Shashank Rajput, Ying Sheng, Sean Owen +1
The size of the key-value (KV) cache plays a critical role in determining both the maximum context length and the number of concurrent requests supported during inference in modern…
LoRA Learns Less and Forgets Less
Dan Biderman, Jacob Portes, Jose Javier Gonzalez Ortiz +9
Low-Rank Adaptation (LoRA) is a widely-used parameter-efficient finetuning method for large language models. LoRA saves memory by training only low rank perturbations to selected w…