1 citations · 3 across the 10 of their papers we have counts for
11 papers
CAM: Question Answering on Entity-Centric Videos with Continuous Extraction and Adaptive Querying
Yizhou Tian, Zizhe Chen, Shiyuan Deng +7
Memory facilitates question answering over long videos by extracting and retrieving facts to fit within the limited context windows of multimodal LLMs (MLLMs). Existing solutions t…
LiteTopK: Exploiting the Curse of Dimensionality for a Fused Indexer-TopK Kernel in Long-Context Sparse Attention
Ziqi Yin, Jianyang Gao, Peiqi Yin +2
Indexer-TopK, which computes scores and selects the top- candidates, is a key primitive in long-context sparse attention for large language models and vector retrieval for recom…
LiveServe: Interaction-Aware Serving for Real-Time Omni-Modal LLMs
Xiangyu Zhi, Peiqi Yin, Sheng Guan +3
Realtime omni-modal LMs support speech-centric conversations where users stream inputs, hear generated audio, and interrupt freely. Existing Omni-LM serving systems still rely on t…
ScaleGANN: Accelerate Large-Scale ANN Indexing by Cost-effective Cloud GPUs
Lan Lu, Peiqi Yin, Isaac Yang +5
Graph-based ANNS algorithms have gained increasing research interest and market adoption due to their efficiency and accuracy in retrieval. Existing approaches primarily rely on CP…
vLLM-Omni: Fully Disaggregated Serving for Any-to-Any Multimodal Models
Peiqi Yin, Jiangyun Zhu, Han Gao +13
Any-to-any multimodal models that jointly handle text, images, video, and audio represent a significant advance in multimodal AI. However, their complex architectures (typically co…
SparseServe: Unlocking Parallelism for Dynamic Sparse Attention in Long-Context LLM Serving
Qihui Zhou, Peiqi Yin, Pengfei Zuo +1
Serving long-context LLMs is costly because attention computation grows linearly with context length. Dynamic sparse attention algorithms (DSAs) mitigate this by attending only to…
Gorgeous: Revisiting the Data Layout for Disk-Resident High-Dimensional Vector Search
Peiqi Yin, Xiao Yan, Qihui Zhou +6
Similarity-based vector search underpins many important applications, but a key challenge is processing massive vector datasets (e.g., in TBs). To reduce costs, some systems utiliz…
PilotANN: Memory-Bounded GPU Acceleration for Vector Search
Yuntao Gui, Peiqi Yin, Xiao Yan +3
Approximate Nearest Neighbor Search (ANNS) has become fundamental to modern deep learning applications, having gained particular prominence through its integration into recent gene…
Progressive Sparse Attention: Algorithm and System Co-design for Efficient Attention in LLM Serving
Qihui Zhou, Peiqi Yin, Pengfei Zuo +1
Processing long contexts has become a critical capability for modern large language models (LLMs). However, serving long-context LLMs comes with significant inference costs due to…
SPT: Fine-Tuning Transformer-based Language Models Efficiently with Sparsification
Yuntao Gui, Xiao Yan, Peiqi Yin +2
Transformer-based large language models (e.g., BERT and GPT) achieve great success, and fine-tuning, which tunes a pre-trained model on a task-specific dataset, is the standard pra…
DGI: Easy and Efficient Inference for GNNs
Peiqi Yin, Xiao Yan, Jinjing Zhou +5
While many systems have been developed to train Graph Neural Networks (GNNs), efficient model inference and evaluation remain to be addressed. For instance, using the widely adopte…