Not all parameters are born equal: Attention is mostly what you need

Nikolay Bogoychev

Transformers are widely used in state-of-the-art machine translation, but the key to their success is still unknown. To gain insight into this, we consider three groups of paramete…