Adding Interpretable Attention to Neural Translation Models Improves Word Alignment

Thomas Zenkel, Joern Wuebker, John DeNero

Multi-layer models with multiple attention heads per layer provide superior translation quality compared to simpler and shallower models, but determining what source context is mos…