Atompack: A Storage and Distribution Layer for Read-Heavy Atomistic ML Training Datasets

Ali Ramlaoui, Daniel T. Speckhard, Sagar Pal +3

Atomistic machine learning datasets are increasingly used for training: large immutable snapshots are read repeatedly, shuffled across epochs, staged across clusters' storage syste…