1 citations · 1 across the 2 of their papers we have counts for
4 papers
Fast Learning Rate Transfer in Shallow Linear Networks at Growing Training Horizons
Mana Sakai, Masaaki Imaizumi
Hyperparameter transfer across model width can substantially reduce the cost of tuning large neural networks, but its behavior when the training horizon grows with width is not ful…
Spectrum-Adaptive Generalization Bounds for Trained Deep Transformers
Mana Sakai, Masaaki Imaizumi
Understanding why trained Transformers generalize well is a fundamental problem in modern machine learning theory, and complexity-based generalization bounds provide a principled w…
Infinite-Width Limit of a Single Attention Layer: Analysis via Tensor Programs
Mana Sakai, Ryo Karakida, Masaaki Imaizumi
In modern theoretical analyses of neural networks, the infinite-width limit is often invoked to justify Gaussian approximations of neuron preactivations (e.g., via neural network G…
Priors for second-order unbiased Bayes estimators
Mana Sakai, Takeru Matsuda, Tatsuya Kubokawa
Asymptotically unbiased priors, introduced by Hartigan (1965), are designed to achieve second-order unbiasedness of Bayes estimators. This paper extends Hartigan's framework to non…