24 citations · 58 across the 50 of their papers we have counts for
3 papers · 1 filter
VarRate: Training-Free Variable-Rate KV Cache Compression for Long-Context LLMs
Shahrzad Esmat, Dhawal Shah, Ali Jannesari
The key-value (KV) cache is the main memory bottleneck in long-context large language model (LLM) inference. Two leading training-free families are both structurally limited: token…
PipeInfer: Accelerating LLM Inference using Asynchronous Pipelined Speculation
Branden Butler, Sixing Yu, Arya Mazaheri +1
Inference of Large Language Models (LLMs) across computer clusters has become a focal point of research in recent times, with many acceleration techniques taking inspiration from C…
Benchmarking LLMs for Environmental Review and Permitting
Rounak Meyur, Hung Phan, Koby Hayashi +12
The National Environment Policy Act (NEPA) stands as a foundational piece of environmental legislation in the United States, requiring federal agencies to consider the environmenta…