540 citations · 4.8k across the 94 of their papers we have counts for
94 papers
Quantifying Hyperparameter Transfer and the Importance of Embedding Layer Learning Rate
Dayal Singh Kalra, Maissam Barkeshli
Hyperparameter transfer allows extrapolating optimal optimization hyperparameters from small to large scales, making it critical for training large language models (LLMs). This is…
Crystalline topological invariants in quantum many-body systems
Naren Manjunath, Maissam Barkeshli
Crystalline symmetries give rise to topological invariants that can distinguish quantum phases of matter. Understanding these in strongly interacting systems is an ongoing research…
Artificial Intelligence and the Structure of Mathematics
Maissam Barkeshli, Michael R. Douglas, Michael H. Freedman
Recent progress in artificial intelligence (AI) is unlocking transformative capabilities for mathematics. There is great hope that AI will help solve major open problems and autono…
On the origin of neural scaling laws: from random graphs to natural language
Maissam Barkeshli, Alberto Alfarano, Andrey Gromov
Scaling laws have played a major role in the modern AI revolution, providing practitioners predictive power over how the model performance will improve with increasing data, comput…
Invariants for (2+1)D bosonic crystalline topological insulators for all 17 wallpaper groups
Vladimir Calvera, Naren Manjunath, Maissam Barkeshli
We study bosonic symmetry-protected topological (SPT) phases in (2+1) dimensions with symmetry , where is a general wallpaper group…
Learning Pseudorandom Numbers with Transformers: Permuted Congruential Generators, Curricula, and Interpretability
Tao Tao, Maissam Barkeshli
We study the ability of Transformer models to learn sequences generated by Permuted Congruential Generators (PCGs), a widely used family of pseudo-random number generators (PRNGs).…