4 papers
Metis: Bridging Text and Code Memory for Self-Evolving Agents
Zijie Dai, Siuhin He, Hui Li +9
Self-evolving agents improve over time by distilling experience from past executions and reusing it in future tasks. Existing systems represent such experience either as natural-la…
SparseServe: Unlocking Parallelism for Dynamic Sparse Attention in Long-Context LLM Serving
Qihui Zhou, Peiqi Yin, Pengfei Zuo +1
Serving long-context LLMs is costly because attention computation grows linearly with context length. Dynamic sparse attention algorithms (DSAs) mitigate this by attending only to…
Gorgeous: Revisiting the Data Layout for Disk-Resident High-Dimensional Vector Search
Peiqi Yin, Xiao Yan, Qihui Zhou +6
Similarity-based vector search underpins many important applications, but a key challenge is processing massive vector datasets (e.g., in TBs). To reduce costs, some systems utiliz…
Progressive Sparse Attention: Algorithm and System Co-design for Efficient Attention in LLM Serving
Qihui Zhou, Peiqi Yin, Pengfei Zuo +1
Processing long contexts has become a critical capability for modern large language models (LLMs). However, serving long-context LLMs comes with significant inference costs due to…