CODA: Rewriting Transformer Blocks as GEMM-Epilogue Programs

Han Guo, Jack Zhang, Arjun Menon +4

Transformer training systems are built around dense linear algebra, yet a nontrivial fraction of end-to-end time is spent on surrounding memory-bound operators. Normalization, acti…