Exponential Family Attention

Kevin Christian Wibisono, Yixin Wang

The self-attention mechanism is the backbone of the transformer neural network underlying most large language models. It can capture complex word patterns and long-range dependenci…