1 citations · 2 across the 3 of their papers we have counts for
4 papers · 1 filter
LIG: Layer-wise Integrated Gradients for Within-Layer Flow Analysis in Transformers
Eight Suzuki, Hideitsu Hino, Noboru Murata
Transformers achieve strong performance, but their internal computations remain opaque. We view each Transformer layer as a dynamic graph whose nodes are token representations and…
Lag Operator SSMs: A Geometric Framework for Structured State Space Modeling
Sutashu Tomonaga, Kenji Doya, Noboru Murata
Structured State Space Models (SSMs), which are at the heart of the recently popular Mamba architecture, are powerful tools for sequence modeling. However, their theoretical founda…
Neural Tangent Kernels and Fisher Information Matrices for Simple ReLU Networks with Random Hidden Weights
Jun'ichi Takeuchi, Yoshinari Takeishi, Noboru Murata +3
Fisher information matrices and neural tangent kernels (NTK) for 2-layer ReLU networks with random hidden weight are argued. We discuss the relation between both notions as a linea…
On a convergence property of a geometrical algorithm for statistical manifolds
Shotaro Akaho, Hideitsu Hino, Noboru Murata
In this paper, we examine a geometrical projection algorithm for statistical inference. The algorithm is based on Pythagorean relation and it is derivative-free as well as represen…