Investigating Neurons and Heads in Transformer-based LLMs for Typographical Errors

Kohei Tsuji, Tatsuya Hiraoka, Yuchang Cheng +2

This paper investigates how LLMs encode inputs with typos. We hypothesize that specific neurons and attention heads recognize typos and fix them internally using local and global c…