1 citations · 1 across the 4 of their papers we have counts for
4 papers
Approximate Spectral Decomposition of Fisher Information Matrix for Simple ReLU Networks
Yoshinari Takeishi, Masazumi Iida, Jun'ichi Takeuchi
We argue the Fisher information matrix (FIM) of one hidden layer networks with the ReLU activation function. For a network, let denote the weight matrix from the $…
An Improved Analysis of Least Squares Superposition Codes with Bernoulli Dictionary
Yoshinari Takeishi, Jun'ichi Takeuchi
For the additive white Gaussian noise channel with average power constraint, sparse superposition codes, proposed by Barron and Joseph in 2010, achieve the capacity. While the code…
A Note on Model Selection for Small Sample Regression
Masanori Kawakita, Jun'ichi Takeuchi
The risk estimator called "Direct Eigenvalue Estimator" (DEE) is studied. DEE was developed for small sample regression. In contrast to many existing model selection criteria, deri…
Minimum Description Length Principle in Supervised Learning with Application to Lasso
Masanori Kawakita, Jun'ichi Takeuchi
The minimum description length (MDL) principle in supervised learning is studied. One of the most important theories for the MDL principle is Barron and Cover's theory (BC theory),…