3 papers
cs.DC2026
EWSJF: An Adaptive Scheduler with Hybrid Partitioning for Mixed-Workload LLM Inference
Bronislav Sidik, Chaya Levi, Joseph Kampeas
Serving Large Language Models (LLMs) under mixed workloads--short, latency-sensitive interactive queries alongside long, throughput-oriented batch requests--poses a fundamental sch…
cs.LG2026
Joint Encoding of KV-Cache Blocks for Scalable LLM Serving
Joseph Kampeas, Emir Haleva
Modern large language models (LLMs) drive interactive AI systems but are bottlenecked by the memory-heavy growth of key-value (KV) caches, which limits real-time throughput under c…
cs.LG2024
Rotation Invariant Quantization for Model Compression
Joseph Kampeas, Yury Nahshan, Hanoch Kremer +4
Post-training Neural Network (NN) model compression is an attractive approach for deploying large, memory-consuming models on devices with limited memory resources. In this study,…