Low Rank Factorization for Compact Multi-Head Self-Attention

Sneha Mehta, Huzefa Rangwala, Naren Ramakrishnan

Effective representation learning from text has been an active area of research in the fields of NLP and text mining. Attention mechanisms have been at the forefront in order to le…