PaPaformer: Language Model from Pre-trained Parallel Paths

Joonas Tapaninaho, Mourad Oussala

The training of modern large-language models requires an increasingly amount of computation power and time. Even smaller variants, such as small-language models (SLMs), take severa…