SimpleBooks: Long-term dependency book dataset with simplified English vocabulary for word-level language modeling

Huyen Nguyen

With language modeling becoming the popular base task for unsupervised representation learning in Natural Language Processing, it is important to come up with new architectures and…