10 papers
TailSieve: Partial-Rollout-Guided Tail Routing for LLM Rollouts
Tianqi Xu, Lu Lv, Haoyang Huang +15
Large-scale rollouts have become a core component of modern LLM systems, spanning reinforcement learning (RL) post-training, on-policy distillation (OPD), and sampling-heavy evalua…
OmniDelta: Skill-Driven Budget Allocation for Token Compression in OmniLLMs
Haoyang Huang, Wenjie Huang, Tianqi Xu +14
Emerging Omni-modal Large Language Models (OmniLLMs) enable unified understanding of text, audio, and video, but their long audio-video token sequences introduce substantial memory…
AutoRPA: Efficient GUI Automation through LLM-Driven Code Synthesis from Interactions
Minghao Chen, Xinyi Hu, Zhou Yu +1
Large Language Model (LLM) based agents have demonstrated proficiency in multi-step interactions with graphical user interfaces (GUIs). While most research focuses on improving sin…
Draft Less, Retrieve More: Hybrid Tree Construction for Speculative Decoding
Yuhao Shen, Tianyu Liu, Xinyi Hu +9
Speculative decoding (SD) accelerates large language model inference by leveraging a draft-then-verify paradigm. To maximize the acceptance rate, recent methods construct expansive…
ECHO: Elastic Speculative Decoding with Sparse Gating for High-Concurrency Scenarios
Xinyi Hu, Yuhao Shen, Baolin Zhang +6
Speculative Decoding promises to accelerate the inference of Large Language Models, yet its efficacy often degrades in production-grade serving. Existing evaluations typically over…
When Hidden States Drift: Can KV Caches Rescue Long-Range Speculative Decoding?
Tianyu Liu, Yuhao Shen, Xinyi Hu +8
Speculative decoding accelerates LLM inference, but SOTA hidden-state-based drafters suffer from long-range decay: draft accuracy degrades as the speculative step increases. Existi…
RoadMapper: A Multi-Agent System for Roadmap Generation of Solving Complex Research Problems
Jiacheng Liu, Zichen Tang, Zhongjun Yang +8
People commonly leverage structured content to accelerate knowledge acquisition and research problem solving. Among these, roadmaps guide researchers through hierarchical subtasks…
Not Search, But Scan: Benchmarking MLLMs on Scan-Oriented Academic Paper Reasoning
Rongjin Li, Zichen Tang, Xianghe Wang +9
With the rapid progress of multimodal large language models (MLLMs), AI already performs well at literature retrieval and certain reasoning tasks, serving as a capable assistant to…
Improving Autoregressive Training with Dynamic Oracles
Jianing Yang, Harshine Visvanathan, Yilin Wang +2
Many tasks within NLP can be framed as sequential decision problems, ranging from sequence tagging to text generation. However, for many tasks, the standard training methods, inclu…
Learning Mutually Informed Representations for Characters and Subwords
Yilin Wang, Xinyi Hu, Matthew R. Gormley
Most pretrained language models rely on subword tokenization, which processes text as a sequence of subword tokens. However, different granularities of text, such as characters, su…