most citedSaving GPU Hours in LLM Inference System Development and Online Workloads with Simulation and DBMS-Inspired Cache Replacement Policies

2 citations · 2 across the 6 of their papers we have counts for

collaborators
Showing cs.PFShow all

1 paper · 1 filter

cs.PF2024★ 2 cited

Saving GPU Hours in LLM Inference System Development and Online Workloads with Simulation and DBMS-Inspired Cache Replacement Policies

Kyoungmin Kim, Jiacheng Li, Kijae Hong +3

LLMs are increasingly used world-wide from daily tasks to agentic systems and data analytics, requiring significant GPU resources. While LLM inference systems are capable of servin…