Technical Manual for a Toolkit for Measuring Contextual Individuation in Transformer Language Models

José Luciano Verçosa Marques, Frederico Jorge Heitmann, Daniel Omar Perez +2

A transformer language model assigns a single, context-independent vector to a word type at its embedding layer, yet is widely believed to individuate that word's occurrences by co…