Meta-Embeddings Based On Self-Attention

Qichen Li, Yuanqing Lin, Luofeng Zhou +1

Creating meta-embeddings for better performance in language modelling has received attention lately, and methods based on concatenation or merely calculating the arithmetic mean of…