5 citations · 12 across the 23 of their papers we have counts for
15 papers · 1 filter
Z-Loss Backward Geometry in Dense Output Heads and Sparse Routers
Bum Jun Kim
Z-loss has been widely applied to the logits of language-model output heads and sparse mixture-of-experts routers. Z-loss constrains the softmax log-normalizers of these output hea…
MiNO: Cotangent-bundle propagator learning for PDEs
Gnankan Landry Regis N'guessan, Bum Jun Kim
Scientific machine learning for partial differential equations commonly targets solution fields, as in physics-informed neural networks, or solution maps, as in neural operators. W…
SEAM: Global consistency beyond local accuracy in scientific machine learning
Gnankan Landry Regis N'guessan, Bum Jun Kim
Scientific machine learning commonly validates models at the level of a subdomain, a benchmark split, or an explanation for one prediction. Yet such local checks cannot establish w…
Looped Transformers with Source-Centered State Evolution
Bum Jun Kim, Kohei Hayashi, Shunsuke Kamiya +3
Looped Transformers create a useful train- and test-time compute axis by reusing the same Transformer block over recurrent depth, increasing effective depth at a fixed parameter co…
Lottery Tickets Are Not Deployment Tickets
Bum Jun Kim
Reports on how sparsification, compression, and lottery tickets change model behavior have been mixed in the prior literature, with beneficial effects observed in some studies and…
SHiPPO: Recurrent Memory with Transported Polynomial Projections
Tomoya Mizuguchi, Bum Jun Kim
HiPPO gives recurrent states memory semantics as coefficients of online polynomial projections, but in fixed channel coordinates. Modern selective SSMs, by contrast, rely on token-…