1 citations · 2 across the 3 of their papers we have counts for
3 papers
cs.LG2024★ 1 cited
KV Cache is 1 Bit Per Channel: Efficient Large Language Model Inference with Coupled Quantization
Tianyi Zhang, Jonah Yi, Zhaozhuo Xu +1
Efficient deployment of Large Language Models (LLMs) requires batching multiple requests together to improve throughput. As the batch size, context length, or model size increases,…
cs.LG2024
NoMAD-Attention: Efficient LLM Inference on CPUs Through Multiply-add-free Attention
Tianyi Zhang, Jonah Wonkyu Yi, Bowen Yao +2
Large language model inference on Central Processing Units (CPU) is challenging due to the vast quantities of expensive Multiply-Add (MAD) matrix operations in the attention comput…
cs.IR2023★ 1 cited
CAPS: A Practical Partition Index for Filtered Similarity Search
Gaurav Gupta, Jonah Yi, Benjamin Coleman +3
With the surging popularity of approximate near-neighbor search (ANNS), driven by advances in neural representation learning, the ability to serve queries accompanied by a set of c…