activity
20242026
most citedScaling Laws Across Model Architectures: A Comparative Analysis of Dense and MoE Models in Large Language Models

1 citations · 1 across the 4 of their papers we have counts for

collaborators
Showing 2025Show all

1 paper · 1 filter

cs.LG2025

Scaling and Transferability of Annealing Strategies in Large Language Model Training

Siqi Wang, Zhengyu Chen, Teng Xiao +5

Learning rate scheduling is crucial for training large language models, yet understanding the optimal annealing strategies across different model configurations remains challenging…