Efficient Softmax Approximation for Deep Neural Networks with Attention Mechanism

Ihor Vasyltsov, Wooseok Chang

There has been a rapid advance of custom hardware (HW) for accelerating the inference speed of deep neural networks (DNNs). Previously, the softmax layer was not a main concern of…