Enhanced QKNorm normalization for neural transformers with the Lp norm

Ezequiel Lopez-Rubio, Javier Montes-Perez, Esteban Jose Palomo

The normalization of query and key vectors is an essential part of the Transformer architecture. It ensures that learning is stable regardless of the scale of these vectors. Some n…