Understanding Transformer Memorization Recall Through Idioms

Adi Haviv, Ido Cohen, Jacob Gidron +3

To produce accurate predictions, language models (LMs) must balance between generalization and memorization. Yet, little is known about the mechanism by which transformer LMs emplo…