1 citations · 5 across the 8 of their papers we have counts for
9 papers
LiveServe: Interaction-Aware Serving for Real-Time Omni-Modal LLMs
Xiangyu Zhi, Peiqi Yin, Sheng Guan +3
Realtime omni-modal LMs support speech-centric conversations where users stream inputs, hear generated audio, and interrupt freely. Existing Omni-LM serving systems still rely on t…
vLLM-Omni: Fully Disaggregated Serving for Any-to-Any Multimodal Models
Peiqi Yin, Jiangyun Zhu, Han Gao +13
Any-to-any multimodal models that jointly handle text, images, video, and audio represent a significant advance in multimodal AI. However, their complex architectures (typically co…
Search-R3: Unifying Reasoning and Embedding in Large Language Models
Yuntao Gui, James Cheng
Despite their remarkable natural language understanding capabilities, Large Language Models (LLMs) have been underutilized for retrieval tasks. We present Search-R3, a novel framew…
SparseServe: Unlocking Parallelism for Dynamic Sparse Attention in Long-Context LLM Serving
Qihui Zhou, Peiqi Yin, Pengfei Zuo +1
Serving long-context LLMs is costly because attention computation grows linearly with context length. Dynamic sparse attention algorithms (DSAs) mitigate this by attending only to…
SAQ: Pushing the Limits of Vector Quantization through Code Adjustment and Dimension Segmentation
Hui Li, Shiyuan Deng, Xiao Yan +2
Approximate Nearest Neighbor Search (ANNS) plays a critical role in applications such as search engines, recommender systems, and RAG for LLMs. Vector quantization (VQ), a crucial…
Gorgeous: Revisiting the Data Layout for Disk-Resident High-Dimensional Vector Search
Peiqi Yin, Xiao Yan, Qihui Zhou +6
Similarity-based vector search underpins many important applications, but a key challenge is processing massive vector datasets (e.g., in TBs). To reduce costs, some systems utiliz…
Towards Efficient and Scalable Distributed Vector Search with RDMA
Xiangyu Zhi, Meng Chen, Xiao Yan +5
Similarity-based vector search facilitates many important applications such as search and recommendation but is limited by the memory capacity and bandwidth of a single machine due…
PilotANN: Memory-Bounded GPU Acceleration for Vector Search
Yuntao Gui, Peiqi Yin, Xiao Yan +3
Approximate Nearest Neighbor Search (ANNS) has become fundamental to modern deep learning applications, having gained particular prominence through its integration into recent gene…
Progressive Sparse Attention: Algorithm and System Co-design for Efficient Attention in LLM Serving
Qihui Zhou, Peiqi Yin, Pengfei Zuo +1
Processing long contexts has become a critical capability for modern large language models (LLMs). However, serving long-context LLMs comes with significant inference costs due to…