5 citations · 8 across the 11 of their papers we have counts for
11 papers
Steering Recurrent Reasoners at Inference Time with Readout Feedback
Shunsuke Kamiya, Masanori Koyama, Seongcheol Jeong +5
Recurrent models, which repeatedly update latent states with shared computation blocks, have emerged as powerful architectures for solving complex reasoning tasks. Existing inferen…
Looped Transformers with Source-Centered State Evolution
Bum Jun Kim, Kohei Hayashi, Shunsuke Kamiya +3
Looped Transformers create a useful train- and test-time compute axis by reusing the same Transformer block over recurrent depth, increasing effective depth at a fixed parameter co…
Anti Mode-Collapse in Mean-Field Transformer via Auxiliary Variables
Masaaki Imaizumi, Masanori Koyama, Noboru Isobe +1
We use a mean-field-based transformer model to theoretically investigate how auxiliary variables, such as positional encoding, prevent mode collapse of self-attention mechanisms. T…
C-voting: Confidence-Based Test-Time Voting without Explicit Energy Functions
Kenji Kubo, Shunsuke Kamiya, Masanori Koyama +3
Neural network models with latent recurrent processing, where identical layers are recursively applied to the latent state, have gained attention as promising models for performing…
Exploration of Fast-Slow Latent Recurrence for Train-Short, Test-Long Generalization
Shota Takashiro, Masanori Koyama, Takeru Miyato +3
We study out of distribution generalization in streaming tasks where models are trained on short sequences but must operate over much longer, unknown horizons under bounded memory.…
Extended Flow Matching: a Method of Conditional Generation with Generalized Continuity Equation
Noboru Isobe, Masanori Koyama, Jinzhe Zhang +2
The task of conditional generation is one of the most important applications of generative models, and numerous methods have been developed to date based on the celebrated flow-bas…
Virtual Human Generative Model: Masked Modeling Approach for Learning Human Characteristics
Kenta Oono, Nontawat Charoenphakdee, Kotatsu Bito +14
Virtual Human Generative Model (VHGM) is a generative model that approximates the joint probability over more than 2000 human healthcare-related attributes. This paper presents the…
Neural Fourier Transform: A General Approach to Equivariant Representation Learning
Masanori Koyama, Kenji Fukumizu, Kohei Hayashi +1
Symmetry learning has proven to be an effective approach for extracting the hidden structure of data, with the concept of equivariance relation playing the central role. However, m…
TabRet: Pre-training Transformer-based Tabular Models for Unseen Columns
Soma Onishi, Kenta Oono, Kohei Hayashi
We present \emph{TabRet}, a pre-trainable Transformer-based model for tabular data. TabRet is designed to work on a downstream task that contains columns not seen in pre-training.…
Fractional SDE-Net: Generation of Time Series Data with Long-term Memory
Kohei Hayashi, Kei Nakagawa
In this paper, we focus on the generation of time-series data using neural networks. It is often the case that input time-series data have only one realized (and usually irregularl…
A Scaling Law for Synthetic-to-Real Transfer: How Much Is Your Pre-training Effective?
Hiroaki Mikami, Kenji Fukumizu, Shogo Murai +5
Synthetic-to-real transfer learning is a framework in which a synthetically generated dataset is used to pre-train a model to improve its performance on real vision tasks. The most…