46 citations · 72 across the 13 of their papers we have counts for
Showing 2024Show all
2 papers · 1 filter
cs.DC2024
A System for Microserving of LLMs
Hongyi Jin, Ruihang Lai, Charlie F. Ruan +5
The recent advances in LLMs bring a strong demand for efficient system support to improve overall serving efficiency. As LLM inference scales towards multiple GPUs and even multipl…
cs.LG2024
WebLLM: A High-Performance In-Browser LLM Inference Engine
Charlie F. Ruan, Yucheng Qin, Akaash R. Parthasarathy +11
Advancements in large language models (LLMs) have unlocked remarkable capabilities. While deploying these models typically requires server-grade GPUs and cloud-based inference, the…