The Conformer Encoder May Reverse the Time Dimension

Robin Schmitt, Albert Zeyer, Mohammad Zeineldeen +2

We sometimes observe monotonically decreasing cross-attention weights in our Conformer-based global attention-based encoder-decoder (AED) models, Further investigation shows that t…