NN-LUT: Neural Approximation of Non-Linear Operations for Efficient Transformer Inference

Joonsang Yu, Junki Park, Seongmin Park +4

Non-linear operations such as GELU, Layer normalization, and Softmax are essential yet costly building blocks of Transformer models. Several prior works simplified these operations…