3 papers
cs.CL2026
A Sovereign, Open-Source Foundation Model for German and English
The Soofi-Team, Soofi-Team, : +31
We present Soofi S 30B-A3B, a sovereign, open-source Mixture-of-Experts (MoE) hybrid Mamba Transformer foundation model for German and English. Its hybrid design activates only 3B…
cs.LG2026
Modalities, a PyTorch-native Framework For Large-scale LLM Training and Research
Max Lübbering, Timm Ruland, Richard Rutmann +8
Today's LLM (pre-) training and research workflows typically allocate a significant amount of compute to large-scale ablation studies. Despite the substantial compute costs of thes…
quant-ph2024
RydbergGPT
David Fitzek, Yi Hong Teoh, Hin Pok Fung +5
We introduce a generative pretained transformer (GPT) designed to learn the measurement outcomes of a neutral atom array quantum computer. Based on a vanilla transformer, our encod…