856 citations · 1.1k across the 51 of their papers we have counts for
52 papers
Fault-tolerant foundation models
Trevor McCourt, Ila R. Fiete, Isaac L. Chuang
Emerging computer hardware often trades reliability for energy efficiency; here we show that large-language models (LLMs) can be trained to tolerate this unreliability, and that ra…
Mean-field theory of rich oscillatory dynamics in low-rank recurrent networks with activity-dependent adaptation
Bowen W. Zheng, Earl K. Miller, Ila R. Fiete
We develop a dynamical mean-field theory for random recurrent networks with low-rank structure and firing-rate-driven adaptation. When the random connectivity is strong enough to g…
B[FM]: Brain Foundation Model via Flow Matching with SplitUNet
Jaedong Hwang, Kathleen Zhang, Wei Dai +5
EEG foundation models can learn generalizable representations from large-scale EEG corpora to enable single-backbone transfer across diverse clinical and brain-computer interface t…
Attractor Geometry of Transformer Memory: From Conflict Arbitration to Confident Hallucination
Qiyao Liang, Risto Miikkulainen, Ila Fiete
Language models draw on two knowledge sources: facts baked into weights (parametric memory, PM) and information in context (working memory, WM). We study two mechanistically distin…
The Blessing of Dimensionality in LLM Fine-tuning: A Variance-Curvature Perspective
Qiyao Liang, Jinyeop Song, Yizhou Liu +4
Weight-perturbation evolution strategies (ES) can fine-tune billion-parameter language models with surprisingly small populations (e.g., ), contradicting classical…
Modular connectivity in neural networks emerges from Poisson noise-motivated regularisation, and promotes robustness and compositional generalisation
Daoyuan Qian, Qiyao Liang, Ila Fiete
Circuits in the brain commonly exhibit modular architectures that factorise complex tasks, resulting in the ability to compositionally generalise and reduce catastrophic forgetting…