A Cost-Efficient FPGA Implementation of Tiny Transformer Model using Neural ODE

Ikumi Okubo, Keisuke Sugiura, Hiroki Matsutani

Transformer has been adopted to image recognition tasks and shown to outperform CNNs and RNNs while it suffers from high training cost and computational complexity. To address thes…