6 citations · 31 across the 19 of their papers we have counts for
21 papers
Which Geometry on Which Layer? Measured Sign-Spectral Assignment in Mixed-Optimizer Training
Hiroki Naganuma, Laura Gomezjurado, Mahdi Ghaznavi +5
Modern neural-network training uses mixed recipes, assigning AdamW-like or sign-based updates to embedding and output layers and spectral updates to hidden matrices. These assignme…
Provable Data Scaling Law for Meta Learning via Complexity Minimization
Kazuto Fukuchi, Ryuichiro Hataya, Kota Matsui
Pre-training has become a fundamental paradigm in modern machine learning, with one of its key empirical benefits being reduced downstream sample complexity as the scale of pre-tra…
Provable Target Sample Complexity Improvements as Pre-Trained Models Scale
Kazuto Fukuchi, Ryuichiro Hataya, Kota Matsui
Pre-trained models have become indispensable for efficiently building models across a broad spectrum of downstream tasks. The advantages of pre-trained models have been highlighted…
Automatic Domain Adaptation by Transformers in In-Context Learning
Ryuichiro Hataya, Kota Matsui, Masaaki Imaizumi
Selecting or designing an appropriate domain adaptation algorithm for a given problem remains challenging. This paper presents a Transformer model that can provably approximate and…
Quantum Circuit -algebra Net
Yuka Hashimoto, Ryuichiro Hataya
This paper introduces quantum circuit -algebra net, which provides a connection between -algebra nets proposed in classical machine learning and quantum circuits. Using $…
Self-attention Networks Localize When QK-eigenspectrum Concentrates
Han Bao, Ryuichiro Hataya, Ryo Karakida
The self-attention mechanism prevails in modern machine learning. It has an interesting functionality of adaptively selecting tokens from an input sequence by modulating the degree…