Design and Implementation of an FPGA-Based Hardware Accelerator for Transformer

Richie Li, Sicheng Chen

Transformer-based large language models (LLMs) rely heavily on intensive matrix multiplications for attention and feed-forward layers, with the Q, K, and V linear projections in th…