Chun-Feng Wu

most citedS

^{3}

: Increasing GPU Utilization during Generative Inference for Higher Throughput

7 citations · 7 across the 1 of their papers we have counts for

collaborators

1 paper

cs.AR2023★ 7 cited

S $^{3}$ : Increasing GPU Utilization during Generative Inference for Higher Throughput

Yunho Jin, Chun-Feng Wu, David Brooks +1

Generating texts with a large language model (LLM) consumes massive amounts of memory. Apart from the already-large model parameters, the key/value (KV) cache that holds informatio…

S3: Increasing GPU Utilization during Generative Inference for Higher Throughput

S $^{3}$ : Increasing GPU Utilization during Generative Inference for Higher Throughput