Efficient Language Modeling for Low-Resource Settings with Hybrid RNN-Transformer Architectures

Gabriel Lindenmaier, Sean Papay, Sebastian Padó

Transformer-based language models have recently been at the forefront of active research in text generation. However, these models' advances come at the price of prohibitive traini…