DCT-Former: Efficient Self-Attention with Discrete Cosine Transform

Carmelo Scribano, Giorgia Franchini, Marco Prato +1

Since their introduction the Trasformer architectures emerged as the dominating architectures for both natural language processing and, more recently, computer vision applications.…