2 papers
cs.LG2026
SelectInfer: Selective Neuron Loading and Computation for On-Device LLMs
Huzaifa Shaaban Kabakibo, Eric Schniedermeyer, Artem Burchanow +1
Large Language Models (LLMs) have demonstrated remarkable capabilities across a range of Natural Language Processing (NLP) tasks, but their high computational and memory demands po…
cs.LG2026
Breaking the Ice: Analyzing Cold Start Latency in vLLM
Huzaifa Shaaban Kabakibo, Animesh Trivedi, Lin Wang
As scalable inference services become popular, the cold start latency of an inference engine becomes important. Today, vLLM has evolved into the de-facto inference engine of choice…