27 citations · 44 across the 12 of their papers we have counts for
Showing 2024Show all
3 papers · 1 filter
cs.LG2024★ 1 cited
Pie: Pooling CPU Memory for LLM Inference
Yi Xu, Ziming Mao, Xiangxi Mo +2
The rapid growth of LLMs has revolutionized natural language processing and AI analysis, but their increasing size and memory demands present significant challenges. A common solut…
cs.AI2024
TurboSpec: Closed-loop Speculation Control System for Optimizing LLM Serving Goodput
Xiaoxuan Liu, Jongseok Park, Langxiang Hu +10
Large Language Model (LLM) serving systems batch concurrent user requests to achieve efficient serving. However, in real-world deployments, such inter-request parallelism from batc…
cs.LG2024★ 6 cited
Optimizing LLM Queries in Relational Data Analytics Workloads
Shu Liu, Asim Biswal, Amog Kamsetty +8
Batch data analytics is a growing application for Large Language Models (LLMs). LLMs enable users to perform a wide range of natural language tasks, such as classification, entity…