11 papers
AgentLongBench: A Controllable Long Benchmark For Long-Contexts Agents via Environment Rollouts
Shicheng Fang, Yuxin Wang, Xiaoran Liu +6
The evolution of Large Language Models (LLMs) into autonomous agents necessitates the management of extensive, dynamic contexts. Current benchmarks, however, remain largely static,…
MrRoPE: Mixed-radix Rotary Position Embedding
Qingyuan Tian, Wenhong Zhu, Xiaoran Liu +2
Rotary Position Embedding (RoPE)-extension refers to modifying or generalizing the Rotary Position Embedding scheme to handle longer sequences than those encountered during pre-tra…
DiRL: An Efficient Post-Training Framework for Diffusion Language Models
Ying Zhu, Jiaxin Wan, Xiaoran Liu +7
Diffusion Language Models (dLLMs) have emerged as promising alternatives to Auto-Regressive (AR) models. While recent efforts have validated their pre-training potential and accele…
Beyond Real: Imaginary Extension of Rotary Position Embeddings for Long-Context LLMs
Xiaoran Liu, Yuerong Song, Zhigeng Liu +6
Rotary Position Embeddings (RoPE) have become a standard for encoding sequence order in Large Language Models (LLMs) by applying rotations to query and key vectors in the complex p…
SIM-CoT: Supervised Implicit Chain-of-Thought
Xilin Wei, Xiaoran Liu, Yuhang Zang +5
Implicit Chain-of-Thought (CoT) methods offer a token-efficient alternative to explicit CoT reasoning in Large Language Models (LLMs), but a persistent performance gap has limited…
Sparse-dLLM: Accelerating Diffusion LLMs with Dynamic Cache Eviction
Yuerong Song, Xiaoran Liu, Ruixiao Li +5
Diffusion Large Language Models (dLLMs) enable breakthroughs in reasoning and parallel decoding but suffer from prohibitive quadratic computational complexity and memory overhead d…