4 papers
SPORK: Self-Speculative Forking to Accelerate Agentic LLM Inference
Huajun Bai, Weiwei Lv, Huichuan Zheng +2
LLM agents are becoming a common interface for research, coding, and question answering, yet their Thought-Action-Observation loop is often serial: the model reasons, emits a tool…
PipeANN-Filter: An Efficient Filtered Vector Search System on SSD
Hao Guo, Jiwu Shu, Youyou Lu
We propose PipeANN-Filter, an efficient filtered vector search system on SSD. Unlike existing systems that explore only valid vectors (i.e., those satisfying the attribute constrai…
Efficient Training on Multiple Consumer GPUs with RoundPipe
Yibin Luo, Shiwei Gao, Huichuan Zheng +2
Fine-tuning Large Language Models (LLMs) on consumer-grade GPUs is highly cost-effective, yet constrained by limited GPU memory and slow PCIe interconnects. Pipeline parallelism co…
SwitchDelta: Asynchronous Metadata Updating for Distributed Storage with In-Network Data Visibility
Junru Li, Qing Wang, Zhe Yang +3
Distributed storage systems typically maintain strong consistency between data nodes and metadata nodes by adopting ordered writes: 1) first installing data; 2) then updating metad…