Fast-FNet: Accelerating Transformer Encoder Models via Efficient Fourier Layers

Nurullah Sevim, Ege Ozan Özyedek, Furkan Şahinuç +1

Transformer-based language models utilize the attention mechanism for substantial performance improvements in almost all natural language processing (NLP) tasks. Similar attention…