Showing cs.LGShow all
2 papers · 1 filter
cs.LG2026
MultiAlloc: Budget-Aware Precision Allocation for KV Cache Quantization
Yangchengyu Zhou, Zhiping Wu, Runchang Jiang +5
The Key-Value (KV) cache constitutes a dominant source of memory consumption during large language model (LLM) inference, particularly in long-context and complex reasoning scenari…
cs.LG2024
ONNXPruner: ONNX-Based General Model Pruning Adapter
Dongdong Ren, Wenbin Li, Tianyu Ding +5
Recent advancements in model pruning have focused on developing new algorithms and improving upon benchmarks. However, the practical application of these algorithms across various…