Transformers are Deep Infinite-Dimensional Non-Mercer Binary Kernel Machines

Matthew A. Wright, Joseph E. Gonzalez

Despite their ubiquity in core AI fields like natural language processing, the mechanics of deep attention-based neural networks like the Transformer model are not fully understood…