From the 1 of 7 linked papers with an AI index.
7 papers
Looped Transformers with Source-Centered State Evolution
Bum Jun Kim, Kohei Hayashi, Shunsuke Kamiya +3
The paper introduces Source‑Centered State Evolution (SCSE), a method for looped Transformers that preserves input conditioning while keeping a fixed reference point, improving rec…
Exploration of Fast-Slow Latent Recurrence for Train-Short, Test-Long Generalization
Shota Takashiro, Masanori Koyama, Takeru Miyato +3
We study out of distribution generalization in streaming tasks where models are trained on short sequences but must operate over much longer, unknown horizons under bounded memory.…
DiffusionBlocks: Block-wise Neural Network Training via Diffusion Interpretation
Makoto Shing, Masanori Koyama, Takuya Akiba
End-to-end backpropagation requires storing activations throughout all layers, creating memory bottlenecks that limit model scalability. Existing block-wise training methods offer…
Anti Mode-Collapse in Mean-Field Transformer via Auxiliary Variables
Masaaki Imaizumi, Masanori Koyama, Noboru Isobe +1
We use a mean-field-based transformer model to theoretically investigate how auxiliary variables, such as positional encoding, prevent mode collapse of self-attention mechanisms. T…
C-voting: Confidence-Based Test-Time Voting without Explicit Energy Functions
Kenji Kubo, Shunsuke Kamiya, Masanori Koyama +3
Neural network models with latent recurrent processing, where identical layers are recursively applied to the latent state, have gained attention as promising models for performing…
Pairwise Optimal Transports for Training All-to-All Flow-Based Condition Transfer Model
Kotaro Ikeda, Masanori Koyama, Jinzhe Zhang +2
In this paper, we propose a flow-based method for learning all-to-all transfer maps among conditional distributions that approximates pairwise optimal transport. The proposed metho…