DsDm: Model-Aware Dataset Selection with Datamodels

Logan Engstrom, Axel Feldmann, Aleksander Madry

When selecting data for training large-scale models, standard practice is to filter for examples that match human notions of data quality. Such filtering yields qualitatively clean…