Lexinvariant Language Models

Qian Huang, Eric Zelikman, Sarah Li Chen +3

Token embeddings, a mapping from discrete lexical symbols to continuous vectors, are at the heart of any language model (LM). However, lexical symbol meanings can also be determine…