On the Optimal Memorization Capacity of Transformers

Tokio Kajitsuka, Issei Sato

Recent research in the field of machine learning has increasingly focused on the memorization capacity of Transformers, but how efficient they are is not yet well understood. We de…