Query-Key Normalization for Transformers

Alex Henry, Prudhvi Raj Dachapally, Shubham Pawar +1

Low-resource language translation is a challenging but socially valuable NLP task. Building on recent work adapting the Transformer's normalization to this setting, we propose QKNo…