Model Internal Sleuthing: Finding Lexical Identity and Inflectional Features in Modern Language Models

Michael Li, Nishant Subramani

Large transformer-based language models dominate modern NLP, yet our understanding of how they encode linguistic information relies primarily on studies of early models like BERT a…