Modalities, a PyTorch-native Framework For Large-scale LLM Training and Research

Max Lübbering, Timm Ruland, Richard Rutmann +8

Today's LLM (pre-) training and research workflows typically allocate a significant amount of compute to large-scale ablation studies. Despite the substantial compute costs of thes…