8 papers
d-HNSW: A High-performance Vector Search Engine on Disaggregated Memory
Fei Fang, Yi Liu, Chen Qian
Efficient vector search is essential for powering large-scale AI applications, such as LLMs. Existing solutions are designed for monolithic architectures where compute and memory a…
PlanetServe: A Decentralized, Scalable, and Privacy-Preserving Overlay for Democratizing Large Language Model Serving
Fei Fang, Yifan Hua, Shengze Wang +4
While significant progress has been made in research and development on open-source and cost-efficient large-language models (LLMs), serving scalability remains a critical challeng…
Trinity: Disaggregating Vector Search from Prefill-Decode Disaggregation in LLM Serving
Yi Liu, Chen Qian
Prefill and decode (PD) disaggregation separates prompt prefill and token-by-token decode stages into distinct GPU pools and has become the dominant architecture for large-scale LL…
Fantasy: Efficient Large-scale Vector Search on GPU Clusters with GPUDirect Async
Yi Liu, Chen Qian
Vector similarity search has become a critical component in AI-driven applications such as large language models (LLMs). To achieve high recall and low latency, GPUs are utilized t…
A Distributed Learned Hash Table
Shengze Wang, Yi Liu, Xiaoxue Zhang +2
Distributed Hash Tables (DHTs) are pivotal in numerous high-impact key-value applications built on distributed networked systems, offering a decentralized architecture that avoids…
Efficient Vector Search on Disaggregated Memory with d-HNSW
Yi Liu, Fei Fang, Chen Qian
Efficient vector query processing is critical to enable AI applications at scale. Recent solutions struggle with growing vector datasets that exceed single-machine memory capacity,…