Widening the Representation Bottleneck in Neural Machine Translation with Lexical Shortcuts

Denis Emelin, Ivan Titov, Rico Sennrich

The transformer is a state-of-the-art neural translation model that uses attention to iteratively refine lexical representations with information drawn from the surrounding context…