3 papers
cs.LG2025
Optimal Scaling Needs Optimal Norm
Oleg Filatov, Jiangtao Wang, Jan Ebert +1
Despite recent progress in optimal hyperparameter transfer under model and dataset scaling, no unifying explanatory principle has been established. For Adam and Scion optimizers, w…
cs.DC2025
Memory and Bandwidth are All You Need for Fully Sharded Data Parallel
Jiangtao Wang, Jan Ebert, Oleg Filatov +1
Transformer models have revolutionized a wide spectrum of disciplines, especially in language processing. The recent success has proven that model size scalability is crucial for a…
physics.comp-ph2025
The Artificial Scientist -- in-transit Machine Learning of Plasma Simulations
Jeffrey Kelling, Vicente Bolea, Michael Bussmann +19
Increasing HPC cluster sizes and large-scale simulations that produce petabytes of data per run, create massive IO and storage challenges for analysis. Deep learning-based techniqu…