Hardware Accelerator for Multi-Head Attention and Position-Wise Feed-Forward in the Transformer

Siyuan Lu, Meiqi Wang, Shuang Liang +2

Designing hardware accelerators for deep neural networks (DNNs) has been much desired. Nonetheless, most of these existing accelerators are built for either convolutional neural ne…