10 papers
From Prediction to Intervention: Personalized Meal-Level Glucose Regulation via an LLM Agent
Mingyu Huang, Weiqing Min, Ying Jin +2
Personalized glucose regulation remains a central yet unresolved challenge in precision nutrition, as postprandial glucose response varies substantially across individuals. Existin…
GQA-μP: The maximal parameterization update for grouped query attention
Kyle R. Chickering, Huijuan Wang, Mengxi Wu +7
Hyperparameter transfer across model architectures dramatically reduces the amount of compute necessary for tuning large language models (LLMs). The maximal update parameterization…
CoRPO: Adding a Correctness Bias to GRPO Improves Generalization
Anisha Garg, Claire Zhang, Nishit Neema +3
Group-Relative Policy Optimization (GRPO) has emerged as the standard for training reasoning capabilities in large language models through reinforcement learning. By estimating adv…
Scaling with Collapse: Efficient and Predictable Training of LLM Families
Shane Bergsma, Bin Claire Zhang, Nolan Dey +3
Effective LLM training depends on predictable scaling of key quantities -- such as final loss and optimal hyperparameters -- with model and dataset size. Qiu et al. (2025) recently…
Predicting Training Re-evaluation Curves Enables Effective Data Curriculums for LLMs
Shane Bergsma, Nolan Dey, Joel Hestness
Data curriculums have become central to successful LLM training, yet principles governing optimal data placement remain unclear. We introduce the *training re-evaluation curve (TRE…
Sparse maximal update parameterization: A holistic approach to sparse training dynamics
Nolan Dey, Shane Bergsma, Joel Hestness
Several challenges make it difficult for sparse neural networks to compete with dense models. First, setting a large fraction of weights to zero impairs forward and gradient signal…