81 citations · 108 across the 6 of their papers we have counts for
Showing 2024Show all
2 papers · 1 filter
cs.LG2024
Is the GPU Half-Empty or Half-Full? Practical Scheduling Techniques for LLMs
Ferdi Kossmann, Bruce Fontaine, Daya Khudia +2
Serving systems for Large Language Models (LLMs) improve throughput by processing several requests concurrently. However, multiplexing hardware resources between concurrent request…
cs.CL2024★ 4 cited
MosaicBERT: A Bidirectional Encoder Optimized for Fast Pretraining
Jacob Portes, Alex Trott, Sam Havens +6
Although BERT-style encoder models are heavily used in NLP research, many researchers do not pretrain their own BERTs from scratch due to the high cost of training. In the past hal…