1 citations · 2 across the 4 of their papers we have counts for
5 papers · 1 filter
LiveServe: Interaction-Aware Serving for Real-Time Omni-Modal LLMs
Xiangyu Zhi, Peiqi Yin, Sheng Guan +3
Realtime omni-modal LMs support speech-centric conversations where users stream inputs, hear generated audio, and interrupt freely. Existing Omni-LM serving systems still rely on t…
vLLM-Omni: Fully Disaggregated Serving for Any-to-Any Multimodal Models
Peiqi Yin, Jiangyun Zhu, Han Gao +13
Any-to-any multimodal models that jointly handle text, images, video, and audio represent a significant advance in multimodal AI. However, their complex architectures (typically co…
SparseServe: Unlocking Parallelism for Dynamic Sparse Attention in Long-Context LLM Serving
Qihui Zhou, Peiqi Yin, Pengfei Zuo +1
Serving long-context LLMs is costly because attention computation grows linearly with context length. Dynamic sparse attention algorithms (DSAs) mitigate this by attending only to…
Towards Efficient and Scalable Distributed Vector Search with RDMA
Xiangyu Zhi, Meng Chen, Xiao Yan +5
Similarity-based vector search facilitates many important applications such as search and recommendation but is limited by the memory capacity and bandwidth of a single machine due…
PilotANN: Memory-Bounded GPU Acceleration for Vector Search
Yuntao Gui, Peiqi Yin, Xiao Yan +3
Approximate Nearest Neighbor Search (ANNS) has become fundamental to modern deep learning applications, having gained particular prominence through its integration into recent gene…