Showing 2026Show all
3 papers · 1 filter
cs.AI2026
PTStore (Prefix Tensor Store): Distributed Prefix Caching and Replication for High Throughput Inference Serving
Meghana Maghyastha, Robert Underwood, Randal Burns +1
Inspired by the design of client caching in Content Delivery Networks (CDNs), PTStore distributes and replicates popular tensors that form reusable KV cache prefixes, which are the…
cs.DC2026
Recency/Frequency Adaptive KV Caching for Large Language Model Serving
Yang Shen, Meghana Madhyastha, Robert Underwood +2
Key-value (KV) caching is a powerful technique for accelerating large language model inference and generation. Inference workloads are large and diverse, which makes them difficult…
cs.LG2026
Vectorized Dynamic Histograms for Sparse Oblique Forests
Ariel Lubonja, Jungsang Yoon, Haoyin Xu +6
Sparse oblique (SPO), part of the top-ranked configuration of Google's Yggdrasil Decision Forests (YDF), improve the accuracy while maintaining interpretability of Random Forests (…