3 papers
cs.DC2025
HACK: Homomorphic Acceleration via Compression of the Key-Value Cache for Disaggregated LLM Inference
Zeyu Zhang, Haiying Shen, Shay Vargaftik +3
Disaggregated Large Language Model (LLM) inference has gained popularity as it separates the computation-intensive prefill stage from the memory-intensive decode stage, avoiding th…
cs.LG2024
Don't Stop Me Now: Embedding Based Scheduling for LLMs
Rana Shahout, Eran Malach, Chunwei Liu +3
Efficient scheduling is crucial for interactive Large Language Model (LLM) applications, where low request completion time directly impacts user engagement. Size-based scheduling a…
cs.DS2024
Learning-Augmented Frequency Estimation in Sliding Windows
Rana Shahout, Ibrahim Sabek, Michael Mitzenmacher
We show how to utilize machine learning approaches to improve sliding window algorithms for approximate frequency estimation problems, under the ``algorithms with predictions'' fra…