From the 1 of 6 linked papers with an AI index.
6 papers
Improving the Robustness/Accuracy Tradeoff Against Adversarial Attacks Using Information Bottleneck Distillation Through Dual Teachers
Vincent Ryusuke Takahashi, Yoshinari Takeishi, Jun'ichi Takeuchi +1
The paper extends information bottleneck distillation by adding a clean‑trained teacher alongside a robust teacher, using cross‑layer attention to improve both clean accuracy and a…
Approximating Simple ReLU Networks based on Spectral Decomposition of Fisher Information
Ka Long Keith Ho, Yoshinari Takeishi, Junichi Takeuchi
Properties of Fisher information matrices of 2-layer neural ReLU networks with random hidden weights are studied. For these networks, it is known that the eigenvalue distribution h…
Dynamical Properties of Dense Associative Memory
Kazushi Mimura, Jun'ichi Takeuchi, Yuto Sumikawa +2
Dense associative memory, a fundamental instance of modern Hopfield networks, can store a large number of memory patterns as equilibrium states of recurrent networks. While the sta…
Neural Tangent Kernels and Fisher Information Matrices for Simple ReLU Networks with Random Hidden Weights
Jun'ichi Takeuchi, Yoshinari Takeishi, Noboru Murata +3
Fisher information matrices and neural tangent kernels (NTK) for 2-layer ReLU networks with random hidden weight are argued. We discuss the relation between both notions as a linea…
Asymptotically Minimax Regret by Bayes Mixtures
Jun'ichi Takeuchi, Andrew R. Barron
We study the problems of data compression, gambling and prediction of a sequence from an alphabet , in terms of regret and expected regret (redundancy)…
Sparse Superposition Codes with Binomial Dictionary are Capacity-Achieving with Maximum Likelihood Decoding
Yoshinari Takeishi, Jun'ichi Takeuchi
It is known that sparse superposition codes asymptotically achieve the channel capacity over the additive white Gaussian noise channel with both maximum likelihood decoding and eff…