4 papers
Why GRPO Needs Normalization: A Local-Curvature Perspective on Adaptive Gradients
Cheng Ge, Caitlyn Heqi Yin, Hao Liang +1
Reinforcement learning (RL) has become a key driver of language model reasoning. Among RL algorithms, Group Relative Policy Optimization (GRPO) is the de facto standard, avoiding t…
Mosaic: Unlocking Long-Context Inference for Diffusion LLMs via Global Memory Planning and Dynamic Peak Taming
Liang Zheng, Bowen Shi, Yitao Hu +5
Diffusion-based large language models (dLLMs) have emerged as a promising paradigm, utilizing simultaneous denoising to enable global planning and iterative refinement. While these…
Toward Unifying Group Fairness Evaluation from a Sparsity Perspective
Zhecheng Sheng, Jiawei Zhang, Enmao Diao
Ensuring algorithmic fairness remains a significant challenge in machine learning, particularly as models are increasingly applied across diverse domains. While numerous fairness c…
Lost in Tokenization: Context as the Key to Unlocking Biomolecular Understanding in Scientific LLMs
Kai Zhuang, Jiawei Zhang, Yumou Liu +10
Scientific Large Language Models (Sci-LLMs) have emerged as a promising frontier for accelerating biological discovery. However, these models face a fundamental challenge when proc…