Scheduling Data to Compute: Hierarchical Data Pipelining for LLM Training on AMD GPUs

Gengyu Rao, Ghadeer Ahmed H Alabandi, Su Ann Chong +28

Large language models (LLM) have been the key focus of recent machine learning breakthroughs. As the model size grows exponentially, the computation needed for training is astronom…