KV-weights are all you need for skipless transformers

Nils Graef

He and Hofmann (arXiv:2311.01906) detailed a skipless transformer without the V and P (post-attention projection) linear layers, which reduces the total number of weights. However,…