Intra-neuronal attention within language models Relationships between activation and semantics

Michael Pichat, William Pogrund, Paloma Pichat +4

This study investigates the ability of perceptron-type neurons in language models to perform intra-neuronal attention; that is, to identify different homogeneous categorical segmen…