NeuroGame Transformer: Gibbs-Inspired Attention Driven by Game Theory and Statistical Physics

Djamel Bouchaffra, Faycal Ykhlef, Hanene Azzag +2

Standard attention mechanisms in transformers are limited by their pairwise formulation, which hinders the modeling of higher-order dependencies among tokens. We introduce the Neur…