4 papers · 1 filter
Teuken-7B-Base & Teuken-7B-Instruct: Towards European LLMs
Mehdi Ali, Michael Fromm, Klaudia Thellmann +38
We present two multilingual LLMs, Teuken 7B-base and Teuken 7B-instruct, designed to embrace Europe's linguistic diversity by supporting all 24 official languages of the European U…
The Artificial Scientist -- in-transit Machine Learning of Plasma Simulations
Jeffrey Kelling, Vicente Bolea, Michael Bussmann +19
Increasing HPC cluster sizes and large-scale simulations that produce petabytes of data per run, create massive IO and storage challenges for analysis. Deep learning-based techniqu…
Memory and Bandwidth are All You Need for Fully Sharded Data Parallel
Jiangtao Wang, Jan Ebert, Oleg Filatov +1
Transformer models have revolutionized a wide spectrum of disciplines, especially in language processing. The recent success has proven that model size scalability is crucial for a…
Time Transfer: On Optimal Learning Rate and Batch Size In The Infinite Data Limit
Oleg Filatov, Jan Ebert, Jiangtao Wang +1
One of the main challenges in optimal scaling of large language models (LLMs) is the prohibitive cost of hyperparameter tuning, particularly learning rate and batch size .…