most citedKVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization

9 citations · 10 across the 3 of their papers we have counts for

collaborators
Showing cs.CLShow all

2 papers · 1 filter

cs.CL2024

Squeezed Attention: Accelerating Long Context Length LLM Inference

Coleman Hooper, Sehoon Kim, Hiva Mohammadzadeh +6

Emerging Large Language Model (LLM) applications require long input context in order to perform complex tasks like document analysis and code generation. For these long context len…

cs.CL2023★ 1 cited

SPEED: Speculative Pipelined Execution for Efficient Decoding

Coleman Hooper, Sehoon Kim, Hiva Mohammadzadeh +4

Generative Large Language Models (LLMs) based on the Transformer architecture have recently emerged as a dominant foundation model for a wide range of Natural Language Processing t…