1 citations · 1 across the 3 of their papers we have counts for
5 papers
KVCompass: A Discovery Plane for Internet-Scale Prefix Cache Sharing in LLM Serving
Shiyi Ling, Fei Fang, Chen Qian
KV cache reuse is one of the most effective ways to reduce LLM inference cost, but today its benefits are largely confined to a single cluster. When a provider serves the same mode…
Tessera: Demand-Driven KV Cache Management for Retrieval-Augmented LLM Serving
Fei Fang, Chung-Hsiang Lo, Yi Liu +2
RAG and retrieval-based agent memory both inject retrieved content into LLM prompts, as document chunks and recalled memory records, respectively. The same content can recur across…
d-HNSW: A High-performance Vector Search Engine on Disaggregated Memory
Fei Fang, Yi Liu, Chen Qian
Efficient vector search is essential for powering large-scale AI applications, such as LLMs. Existing solutions are designed for monolithic architectures where compute and memory a…
Efficient Vector Search on Disaggregated Memory with d-HNSW
Yi Liu, Fei Fang, Chen Qian
Efficient vector query processing is critical to enable AI applications at scale. Recent solutions struggle with growing vector datasets that exceed single-machine memory capacity,…
PlanetServe: A Decentralized, Scalable, and Privacy-Preserving Overlay for Democratizing Large Language Model Serving
Fei Fang, Yifan Hua, Shengze Wang +4
While significant progress has been made in research and development on open-source and cost-efficient large-language models (LLMs), serving scalability remains a critical challeng…