Showing cs.CLShow all
3 papers · 1 filter
cs.CL2026
Mixture of Training: Recombining Small-Scale Scaffolded Pretraining Runs into a Larger Language Model
Mohammed Sabry, Sean Augenstein, Keith Rush +1
We ask whether language-model pre-training can be decomposed into smaller, independently trainable jobs that can later be recomposed into a coherent larger model. We introduce Mixt…
cs.CL2026
Induction Signatures Are Not Enough: A Matched-Compute Study of Load-Bearing Structure in In-Context Learning
Mohammed Sabry, Anya Belz
Mechanism-targeted synthetic data is increasingly proposed as a way to steer pretraining toward desirable capabilities, but it remains unclear how such interventions should be eval…
cs.CL2025
ATHAR: A High-Quality and Diverse Dataset for Classical Arabic to English Translation
Mohammed Khalil, Mohammed Sabry
Classical Arabic represents a significant era that encompasses the golden age of Arab culture, philosophy, and scientific literature. With a broad consensus on the importance of tr…