3 papers
cs.LG2026
Smaller Models are Natural Explorers for Policy-Level Diversity in GRPO
Yiming Ren, Yiran Xu, Zicheng Lin +8
We identify a new dimension for enhancing rollout diversity in Group Relative Policy Optimization (GRPO) for LLMs. While GRPO relies on diverse rollouts, prevailing strategies prim…
cs.AI2026
daVinci-LLM:Towards the Science of Pretraining
Yiwei Qin, Yixiu Liu, Tiantian Mi +12
The foundational pretraining phase determines a model's capability ceiling, as post-training struggles to overcome capability foundations established during pretraining, yet it rem…
cs.LG2025
Linear Attention Sequence Parallelism
Weigao Sun, Zhen Qin, Dong Li +3
Sequence parallelism (SP) serves as a prevalent strategy to handle long sequences that exceed the memory limit of a single device. However, for linear sequence modeling methods lik…