3 papers
cond-mat.dis-nn2026
Phase structure of the Random Language Model
Alessio Giorlandino, Eric De Giuli, Sebastian Goldt
Context-free grammars are minimal models of hierarchical structure in human language, generating structured text from recursive production rules. The Random Language Model (RLM) [D…
stat.ML2026
Factual recall in linear associative memories: sharp asymptotics and mechanistic insights
Alessio Giorlandino, Sebastian Goldt, Antoine Maillard
Large language models demonstrate remarkable ability in factual recall, yet the fundamental limits of storing and retrieving input--output associations with neural networks remain…
stat.ML2025
Two failure modes of deep transformers and how to avoid them: a unified theory of signal propagation at initialisation
Alessio Giorlandino, Sebastian Goldt
Finding the right initialisation for neural networks is crucial to ensure smooth training and good performance. In transformers, the wrong initialisation can lead to one of two fai…