On Self-improving Token Embeddings

Mario M. Kubek, Shiraj Pokharel, Thomas Böhme +3

This article introduces a novel and fast method for refining pre-trained static word or, more generally, token embeddings. By incorporating the embeddings of neighboring tokens in…