14 papers
Draft in Parallel, Condition Through Depth: Adjacent Causal Injection for Speculative Decoding
Haohui Zhang, Keyu Chen, Haocheng Sun +4
Parallel speculative drafting generates multiple candidates in one backbone pass, but independent token selection can produce inconsistent continuations that shorten the accepted p…
RSPO: Reward-Swap Policy Optimization for Multi-Turn LLM Agents
Qiang Liu, Taian Guo, Ruizhi Qiao +1
Reinforcement learning holds significant potential for training large language models (LLMs) to handle multi-turn interactive tasks. However, in long-horizon, multi-turn tasks char…
Training-Free Hashing-Based Attention via Binary Principal Components
Daohai Yu, Zhanpeng Zeng, Keyu Chen +6
Long-context large language models (LLMs) are increasingly deployed in real-world applications, yet self-attention remains a major efficiency bottleneck -- especially during decodi…
Breaking the Evaluation Paradox: Evaluating High-Entropy Search with Computationally Irreducible Constraints
Juntao Wu, Wei Wen, Xianting Huang +4
Evaluating the exhaustive search capabilities of large language models (LLMs) is plagued by a fundamental paradox: verifying completeness requires complete ground truth, yet high-e…
Beyond Encoder Accumulation: Measuring Encoder Roles in Multi-Encoder VLMs
Wei Ding, Yudong Zhang, Ruobing Xie +3
As foundation models scale toward fusing more heterogeneous visual streams, understanding how diverse encoders interact under joint training becomes a prerequisite for principled d…
Skill Is Not Document: Query-Conditioned Compatibility for LLM Agent Skill Routing
Zifei Wang, Wei Wen, Qiang Ji +3
Large language model agents increasingly rely on reusable skills, making skill retrieval a critical front-end component of agent systems. Skill retrieval, however, is not ordinary…
Moment-Video: Diagnosing Temporal Fidelity of Video MLLMs on Momentary Visual Events
Xiaolin Liu, Yilun Zhu, Xiangyu Zhao +9
Video multimodal large language models (MLLMs) have made rapid progress in general and long-form video understanding, yet their ability to capture and use brief, answer-critical vi…
DenseMLLM: Standard Multimodal LLMs for Dense Prediction
Yi Li, Hongze Shen, Lexiang Tang +6
Multimodal Large Language Models (MLLMs) have demonstrated exceptional capabilities in high-level visual understanding. However, extending these models to fine-grained dense predic…
When should I search more: Adaptive Complex Query Optimization with Reinforcement Learning
Wei Wen, Sihang Deng, Tianjun Wei +3
Query optimization is a crucial component for the efficacy of Retrieval-Augmented Generation (RAG) systems. While reinforcement learning (RL)-based agentic and reasoning methods ha…
Adaptive Dual Reasoner: Large Reasoning Models Can Think Efficiently by Hybrid Reasoning
Yujian Zhang, Keyu Chen, Zhifeng Shen +2
Although Long Reasoning Models (LRMs) have achieved superior performance on various reasoning scenarios, they often suffer from increased computational costs and inference latency…
HiChunk: Evaluating and Enhancing Retrieval-Augmented Generation with Hierarchical Chunking
Wensheng Lu, Keyu Chen, Ruizhi Qiao +1
Retrieval-Augmented Generation (RAG) enhances the response capabilities of language models by integrating external knowledge sources. However, document chunking as an important par…
ASPD: Unlocking Adaptive Serial-Parallel Decoding by Exploring Intrinsic Parallelism in LLMs
Keyu Chen, Zhifeng Shen, Daohai Yu +5
The increasing scale and complexity of large language models (LLMs) pose significant inference latency challenges, primarily due to their autoregressive decoding paradigm character…
RocketEval: Efficient Automated LLM Evaluation via Grading Checklist
Tianjun Wei, Wei Wen, Ruizhi Qiao +2
Evaluating large language models (LLMs) in diverse and challenging scenarios is essential to align them with human preferences. To mitigate the prohibitive costs associated with hu…
Multimodal Label Relevance Ranking via Reinforcement Learning
Taian Guo, Taolin Zhang, Haoqian Wu +3
Conventional multi-label recognition methods often focus on label confidence, frequently overlooking the pivotal role of partial order relations consistent with human preference. T…