From the 2 of 12 linked papers with an AI index.
12 papers
OmniDelta: Skill-Driven Budget Allocation for Token Compression in OmniLLMs
Haoyang Huang, Wenjie Huang, Tianqi Xu +14
OmniDelta is a training-free framework that dynamically allocates token budgets for audio and video inputs in omni-modal large language models, using skill pools and local complexi…
D-cut: Adaptive Verification Depth Pruning for Batched Speculative Decoding
Tianyu Liu, Yuhao Shen, Rui Cen +7
The paper introduces D-Cut, an adaptive method that prunes draft tokens across a batch to focus verification on the most promising tokens, improving the speed of speculative decodi…
Breaking Entropy Bounds: Accelerating RL Training via MTP with Rejection Sampling
Yucheng Li, Huiqiang Jiang, Yang Xu +14
Reinforcement learning (RL) has become a key component in modern large language models, yet the rollout stage remains the key bottleneck in RL training pipelines. Although Multi-To…
Maestro: Reinforcement Learning to Orchestrate Hierarchical Model-Skill Ensembles
Jinyang Wu, Guocheng Zhai, Ruihan Jin +7
The proliferation of large language models (LLMs) and modular skills has endowed autonomous agents with increasingly powerful capabilities. Existing frameworks typically rely on mo…
Draft Less, Retrieve More: Hybrid Tree Construction for Speculative Decoding
Yuhao Shen, Tianyu Liu, Xinyi Hu +9
Speculative decoding (SD) accelerates large language model inference by leveraging a draft-then-verify paradigm. To maximize the acceptance rate, recent methods construct expansive…
ECHO: Elastic Speculative Decoding with Sparse Gating for High-Concurrency Scenarios
Xinyi Hu, Yuhao Shen, Baolin Zhang +6
Speculative Decoding promises to accelerate the inference of Large Language Models, yet its efficacy often degrades in production-grade serving. Existing evaluations typically over…