15 citations · 26 across the 7 of their papers we have counts for
Showing cs.LGShow all
2 papers · 1 filter
cs.LG2024
Is the GPU Half-Empty or Half-Full? Practical Scheduling Techniques for LLMs
Ferdi Kossmann, Bruce Fontaine, Daya Khudia +2
Serving systems for Large Language Models (LLMs) improve throughput by processing several requests concurrently. However, multiplexing hardware resources between concurrent request…
cs.LG2022★ 3 cited
Optimizing Mixture of Experts using Dynamic Recompilations
Ferdinand Kossmann, Zhihao Jia, Alex Aiken
The Mixture of Experts architecture allows for outrageously large neural networks by scaling model parameter size independently from computational demand (FLOPs). However, current…