10 papers
Relevant but Incomplete: Referential Dangling as a Paradigm-Level Failure Mode in Hard Prompt Compression
Zhengpei Hu, Kai Li, Dapeng Fu +5
Hard prompt compression reduces long-context inference cost by independently scoring tokens, sentences, or chunks and retaining the highest-scoring units under a budget. We identif…
OmniDelta: Skill-Driven Budget Allocation for Token Compression in OmniLLMs
Haoyang Huang, Wenjie Huang, Tianqi Xu +14
OmniDelta is a training-free framework that dynamically allocates token budgets for audio and video inputs in omni-modal large language models, using skill pools and local complexi…
A First-Principles Derivation of LLM Policy Optimization: From Expected Reward to GRPO and Its Structural Extensions
Jianghan Shen, Siqi Luo, Yue Li +9
Policy gradient algorithms for language models optimize the same objective , which has exactly two factors: the trajectory probability…
Reflect-Guard: Enhancing LLM Safeguards against Adversarial Prompts via Logical Self-Reflection
Lixing Lin, Juli You, Yue Li +4
Large language model (LLM) safety classifiers such as Llama Guard are effective at detecting overtly harmful prompts but remain vulnerable to adversarial jailbreak attacks that dis…
Draft Less, Retrieve More: Hybrid Tree Construction for Speculative Decoding
Yuhao Shen, Tianyu Liu, Xinyi Hu +9
Speculative decoding (SD) accelerates large language model inference by leveraging a draft-then-verify paradigm. To maximize the acceptance rate, recent methods construct expansive…
BEAVER: A Training-Free Hierarchical Prompt Compression Method via Structure-Aware Page Selection
Zhengpei Hu, Kai Li, Dapeng Fu +4
The exponential expansion of context windows in LLMs has unlocked capabilities for long-document understanding but introduced severe bottlenecks in inference latency and informatio…