4 papers
d-HNSW: A High-performance Vector Search Engine on Disaggregated Memory
Fei Fang, Yi Liu, Chen Qian
Efficient vector search is essential for powering large-scale AI applications, such as LLMs. Existing solutions are designed for monolithic architectures where compute and memory a…
Trinity: Disaggregating Vector Search from Prefill-Decode Disaggregation in LLM Serving
Yi Liu, Chen Qian
Prefill and decode (PD) disaggregation separates prompt prefill and token-by-token decode stages into distinct GPU pools and has become the dominant architecture for large-scale LL…
Fantasy: Efficient Large-scale Vector Search on GPU Clusters with GPUDirect Async
Yi Liu, Chen Qian
Vector similarity search has become a critical component in AI-driven applications such as large language models (LLMs). To achieve high recall and low latency, GPUs are utilized t…
PartialLoading: User Scheduling and Bandwidth Allocation for Parameter-sharing Edge Inference
Guanqiao Qu, Qian Chen, Xianhao Chen +2
By provisioning inference offloading services, edge inference drives the rapid growth of AI applications at network edge. However, how to reduce the inference latency remains a sig…