16 papers
Pre-Trained Policy Discriminators are General Reward Models
Shihan Dou, Shichun Liu, Yuming Yang +19
We offer a novel perspective on reward modeling by formulating it as a policy discriminator, which quantifies the difference between two policies to generate a reward signal, guidi…
Beyond Real: Imaginary Extension of Rotary Position Embeddings for Long-Context LLMs
Xiaoran Liu, Yuerong Song, Zhigeng Liu +6
Rotary Position Embeddings (RoPE) have become a standard for encoding sequence order in Large Language Models (LLMs) by applying rotations to query and key vectors in the complex p…
LongLLaDA: Unlocking Long Context Capabilities in Diffusion LLMs
Xiaoran Liu, Yuerong Song, Zhigeng Liu +4
Large Language Diffusion Models, or diffusion LLMs, have emerged as a significant focus in NLP research, with substantial effort directed toward understanding their scalability and…
Sparse-dLLM: Accelerating Diffusion LLMs with Dynamic Cache Eviction
Yuerong Song, Xiaoran Liu, Ruixiao Li +5
Diffusion Large Language Models (dLLMs) enable breakthroughs in reasoning and parallel decoding but suffer from prohibitive quadratic computational complexity and memory overhead d…
Towards Economical Inference: Enabling DeepSeek's Multi-Head Latent Attention in Any Transformer-based LLMs
Tao Ji, Bin Guo, Yuanbin Wu +6
Multi-head Latent Attention (MLA) is an innovative architecture proposed by DeepSeek, designed to ensure efficient and economical inference by significantly compressing the Key-Val…
CritiQ: Mining Data Quality Criteria from Human Preferences
Honglin Guo, Kai Lv, Qipeng Guo +8
Language model heavily depends on high-quality data for optimal performance. Existing approaches rely on manually designed heuristics, the perplexity of existing models, training c…