From the 1 of 5 linked papers with an AI index.
5 papers
Looped Transformers with Source-Centered State Evolution
Bum Jun Kim, Kohei Hayashi, Shunsuke Kamiya +3
The paper introduces Source‑Centered State Evolution (SCSE), a method for looped Transformers that preserves input conditioning while keeping a fixed reference point, improving rec…
Exploration of Fast-Slow Latent Recurrence for Train-Short, Test-Long Generalization
Shota Takashiro, Masanori Koyama, Takeru Miyato +3
We study out of distribution generalization in streaming tasks where models are trained on short sequences but must operate over much longer, unknown horizons under bounded memory.…
Anti Mode-Collapse in Mean-Field Transformer via Auxiliary Variables
Masaaki Imaizumi, Masanori Koyama, Noboru Isobe +1
We use a mean-field-based transformer model to theoretically investigate how auxiliary variables, such as positional encoding, prevent mode collapse of self-attention mechanisms. T…
C-voting: Confidence-Based Test-Time Voting without Explicit Energy Functions
Kenji Kubo, Shunsuke Kamiya, Masanori Koyama +3
Neural network models with latent recurrent processing, where identical layers are recursively applied to the latent state, have gained attention as promising models for performing…
Virtual Human Generative Model: Masked Modeling Approach for Learning Human Characteristics
Kenta Oono, Nontawat Charoenphakdee, Kotatsu Bito +14
Virtual Human Generative Model (VHGM) is a generative model that approximates the joint probability over more than 2000 human healthcare-related attributes. This paper presents the…