Efficient GPT Model Pre-training using Tensor Train Matrix Representation

Viktoriia Chekalina, Georgii Novikov, Julia Gusak +2

Large-scale transformer models have shown remarkable performance in language modelling tasks. However, such models feature billions of parameters, leading to difficulties in their…