2 papers
cs.CL2024
Theoretical Analysis of Hierarchical Language Recognition and Generation by Transformers without Positional Encoding
Daichi Hayakawa, Issei Sato
In this study, we provide constructive proof that Transformers can recognize and generate hierarchical language efficiently with respect to model size, even without the need for a…
cs.LG2024
On the Optimal Memorization Capacity of Transformers
Tokio Kajitsuka, Issei Sato
Recent research in the field of machine learning has increasingly focused on the memorization capacity of Transformers, but how efficient they are is not yet well understood. We de…