Do Efficient Transformers Really Save Computation?

Kai Yang, Jan Ackermann, Zhenyu He +6

As transformer-based language models are trained on increasingly large datasets and with vast numbers of parameters, finding more efficient alternatives to the standard Transformer…