4 papers
Bole: Efficient Tree Speculation for Hybrid-Attention Language Models
Li Wang, Yi Su, Xiabao Wu +9
Hybrid-attention large language models combine full attention with recurrent linear attention to reduce long-context inference costs, yet their autoregressive decoding remains memo…
Decoupled Attention Fusion: Accelerating RAG with Efficient KV Cache Reuse
Xiabao Wu, Wentao Liu, Yongchao Liu +1
Retrieval-Augmented Generation (RAG) effectively mitigates hallucinations in Large Language Models (LLMs) but suffers from prohibitive Time-To-First-Token (TTFT) latency in long-co…
AutoRAGTuner: A Declarative Framework for Automatic Optimization of RAG Pipelines
Xintan Zeng, Yongchao Liu, Yice Luo +1
Retrieval-Augmented Generation (RAG) enhances LLMs, but performance is highly sensitive to complex architecture designs and hyper-parameter configurations, which currently rely on…
Scaling Graph Chain-of-Thought Reasoning: A Multi-Agent Framework with Efficient LLM Serving
Chengying Huan, Ziheng Meng, Yongchao Liu +11
Graph Chain-of-Thought (Graph-CoT) enables large language models (LLMs) to perform step-by-step reasoning over graph-structured knowledge, but existing pipelines suffer from low ac…