Scaling Studies for Efficient Parameter Search and Parallelism for Large Language Model Pre-training

Michael Benington, Leo Phan, Chris Pierre Paul +5

AI accelerator processing capabilities and memory constraints largely dictate the scale in which machine learning workloads (e.g., training and inference) can be executed within a…