4 citations · 6 across the 8 of their papers we have counts for
Showing cs.LGShow all
3 papers · 1 filter
cs.LG2025
Optimal Scaling Needs Optimal Norm
Oleg Filatov, Jiangtao Wang, Jan Ebert +1
Despite recent progress in optimal hyperparameter transfer under model and dataset scaling, no unifying explanatory principle has been established. For Adam and Scion optimizers, w…
cs.LG2024
Time Transfer: On Optimal Learning Rate and Batch Size In The Infinite Data Limit
Oleg Filatov, Jan Ebert, Jiangtao Wang +1
One of the main challenges in optimal scaling of large language models (LLMs) is the prohibitive cost of hyperparameter tuning, particularly learning rate and batch size . W…
cs.LG2023★ 4 cited
Tokenizer Choice For LLM Training: Negligible or Crucial?
Mehdi Ali, Michael Fromm, Klaudia Thellmann +18
The recent success of Large Language Models (LLMs) has been predominantly driven by curating the training dataset composition, scaling of model architectures and dataset sizes and…