4 papers
Learning reshapes power-law anisotropy in internal representations
Asahi Nakamuta, Jun-nosuke Teramae
Power-law anisotropy in internal representations has been observed across a wide range of biological and artificial neural systems, from state-of-the-art language models to the mou…
Spectral density of correlated random matrices and nonmonotonic stability in hetero-associative memory networks
Arata Tomoto, Jun-nosuke Teramae
Random matrix theory, which characterizes spectral distributions of infinitely large matrices, plays a central role across diverse fields, including high-dimensional data analysis,…
The Impact of Anisotropic Covariance Structure on the Training Dynamics and Generalization Error of Linear Networks
Taishi Watanabe, Ryo Karakida, Jun-nosuke Teramae
The success of deep neural networks largely depends on the statistical structure of the training data. While learning dynamics and generalization on isotropic data are well-establi…
Dynamical mean-field theory for a highly heterogeneous neural population
Futa Tomita, Jun-nosuke Teramae
Large-scale systems with inherent heterogeneity often exhibit complex dynamics that are crucial for their functional properties. However, understanding how such heterogeneity shape…