Train Once, Answer All: Many Pretraining Experiments for the Cost of One

Sebastian Bordt, Martin Pawelczyk

Recent work has demonstrated that controlled pretraining experiments are a powerful tool for studying the relationship between training data and large language model (LLM) behavior…