2 papers
cs.LG2026
MiniPIC: Flexible Position-Independent Caching in <100LOC
Nathan Ordonez, Thomas Parnell
Retrieval-augmented and agentic workloads repeatedly prefill recurring predictable structured inputs (which we call "spans") such as documents and code files. Yet, prefix caching i…
cs.AI2025
Using Span Queries to Optimize for Cache and Attention Locality
Paul Castro, Nick Mitchell, Nathan Ordonez +3
Clients are evolving beyond chat completion, and now include a variety of innovative inference-time scaling and deep reasoning techniques. At the same time, inference servers remai…