Shift-Accumulate Attention: Multiplier-Free Query--Key Products for Transformer Decoding

Khubaib Ahmed, Amna Noor, Ahsan Ul haq

Power-of-two (PoT) quantisation turns a multiplication into a bit shift, so far only for the post-softmax attention--value product. The earlier and larger product, S=QK^T, has not…