3 papers
cs.LG2025
Loquetier: A Virtualized Multi-LoRA Framework for Unified LLM Fine-tuning and Serving
Yuchen Zhang, Hanyue Du, Chun Cao +1
Low-Rank Adaptation (LoRA) has become a widely adopted parameter-efficient fine-tuning (PEFT) technique for adapting large language models (LLMs) to downstream tasks. While prior w…
cs.CL2025
Adamas: Hadamard Sparse Attention for Efficient Long-Context Inference
Siyuan Yan, Guo-Qing Jiang, Yuchen Zhang +4
Large language models (LLMs) now support context windows of hundreds of thousands to millions of tokens, enabling applications such as long-document summarization, large-scale code…
cs.LG2025
Long-Context Attention Benchmark: From Kernel Efficiency to Distributed Context Parallelism
Tao Bu, Qiangang Wang, Bowen Zeng +4
Transformer-based large language models (LLMs) have achieved remarkable success, yet their standard attention mechanism incurs quadratic computation and memory costs with respect t…