Cosine Normalization: Using Cosine Similarity Instead of Dot Product in Neural Networks

Chunjie Luo, Jianfeng Zhan, Lei Wang +1

Traditionally, multi-layer neural networks use dot product between the output vector of previous layer and the incoming weight vector as the input to activation function. The resul…