4 citations · 7 across the 6 of their papers we have counts for
3 papers · 1 filter
Neural Dimensional Analysis: Foundations of Dimensionless and Scale-Consistent Computation
Bum Jun Kim, Makoto Kawano, Yusuke Iwasawa +1
Modern Transformers rely on many scale-control mechanisms, including normalization, residual scaling, attention temperatures, loss weighting, and adaptive updates. To explain how t…
QuadNorm: Resolution-Robust Normalization for Neural Operators
Bum Jun Kim, Makoto Kawano, Yusuke Iwasawa +1
Normalization layers in neural operators usually compute statistics by uniformly averaging discrete grid values, making the normalization itself discretization-dependent and thereb…
Residual Koopman Spectral Profiling for Predicting and Preventing Transformer Training Instability
Bum Jun Kim, Shohei Taniguchi, Makoto Kawano +2
Training divergence in transformers wastes compute, yet practitioners discover instability only after expensive runs begin. They therefore need an expected probability of failure f…