NNTile: a machine learning framework capable of training extremely large GPT language models on a single node

Aleksandr Mikhalev, Aleksandr Katrutsa, Konstantin Sozykin +1

This study presents an NNTile framework for training large deep neural networks in heterogeneous clusters. The NNTile is based on a StarPU library, which implements task-based para…