Showing cs.LGShow all
3 papers · 1 filter
cs.LG2026
RAP: KV-Cache Compression via RoPE-Aligned Pruning
Jihao Xin, Tian Lyu, David Keyes +2
Long-context inference in large language models (LLMs) is bottlenecked by the memory and compute of the key-value (KV) cache. Structured pruning is a direct way to shrink it: dropp…
cs.LG2024
Towards a Flexible and High-Fidelity Approach to Distributed DNN Training Emulation
Banruo Liu, Mubarak Adetunji Ojewale, Yuhan Ding +1
We propose NeuronaBox, a flexible, user-friendly, and high-fidelity approach to emulate DNN training workloads. We argue that to accurately observe performance, it is possible to e…
cs.LG2023
Kimad: Adaptive Gradient Compression with Bandwidth Awareness
Jihao Xin, Ivan Ilin, Shunkang Zhang +2
In distributed training, communication often emerges as a bottleneck. In response, we introduce Kimad, a solution that offers adaptive gradient compression. By consistently monitor…