Dynamic Evaluation of Transformer Language Models

Ben Krause, Emmanuel Kahembwe, Iain Murray +1

This research note combines two methods that have recently improved the state of the art in language modeling: Transformers and dynamic evaluation. Transformers use stacked layers…