2 citations · 2 across the 8 of their papers we have counts for
4 papers · 1 filter
TailSieve: Partial-Rollout-Guided Tail Routing for LLM Rollouts
Tianqi Xu, Lu Lv, Haoyang Huang +15
Large-scale rollouts have become a core component of modern LLM systems, spanning reinforcement learning (RL) post-training, on-policy distillation (OPD), and sampling-heavy evalua…
OmniDelta: Skill-Driven Budget Allocation for Token Compression in OmniLLMs
Haoyang Huang, Wenjie Huang, Tianqi Xu +14
Emerging Omni-modal Large Language Models (OmniLLMs) enable unified understanding of text, audio, and video, but their long audio-video token sequences introduce substantial memory…
When Hidden States Drift: Can KV Caches Rescue Long-Range Speculative Decoding?
Tianyu Liu, Yuhao Shen, Xinyi Hu +8
Speculative decoding accelerates LLM inference, but SOTA hidden-state-based drafters suffer from long-range decay: draft accuracy degrades as the speculative step increases. Existi…
CacheRAG: A Semantic Caching System for Retrieval-Augmented Generation in Knowledge Graph Question Answering
Yushi Sun, Lei Chen
The integration of Large Language Models (LLMs) with Retrieval-Augmented Generation (RAG) has significantly advanced Knowledge Graph Question Answering (KGQA). However, existing LL…