Showing cs.DCShow all
2 papers · 1 filter
cs.DC2026
vSkipper: Translating Dynamic Layer Skipping into LLM Serving Gains
Wei Da, Yavuz Ferhatosmanoglu, Evangelia Kalyvianaki
Dynamic layer skipping reduces LLM computation by allowing each token to execute only a subset of the model's layers. However, existing skippers rely on specialized generation loop…
cs.DC2025
Astrolabe: Balancing Load in LLM Serving with Randomized Prediction-Guided Scheduling
Wei Da, Evangelia Kalyvianaki
This paper presents Astrolabe, a randomized prediction-guided scheduler for one-shot request dispatch in multi-instance large language model (LLM) serving. Astrolabe improves load…