1 citations · 1 across the 7 of their papers we have counts for
7 papers
Improving the Robustness/Accuracy Tradeoff Against Adversarial Attacks Using Information Bottleneck Distillation Through Dual Teachers
Vincent Ryusuke Takahashi, Yoshinari Takeishi, Jun'ichi Takeuchi +1
Deep neural networks (DNNs) have achieved remarkable success in classical machine learning problems. However, they are known to be vulnerable to adversarial attacks. Countermeasure…
Neural Tangent Kernels and Fisher Information Matrices for Simple ReLU Networks with Random Hidden Weights
Jun'ichi Takeuchi, Yoshinari Takeishi, Noboru Murata +3
Fisher information matrices and neural tangent kernels (NTK) for 2-layer ReLU networks with random hidden weight are argued. We discuss the relation between both notions as a linea…
Approximating Simple ReLU Networks based on Spectral Decomposition of Fisher Information
Ka Long Keith Ho, Yoshinari Takeishi, Junichi Takeuchi
Properties of Fisher information matrices of 2-layer neural ReLU networks with random hidden weights are studied. For these networks, it is known that the eigenvalue distribution h…
Sparse Superposition Codes with Binomial Dictionary are Capacity-Achieving with Maximum Likelihood Decoding
Yoshinari Takeishi, Jun'ichi Takeuchi
It is known that sparse superposition codes asymptotically achieve the channel capacity over the additive white Gaussian noise channel with both maximum likelihood decoding and eff…
Risk Bounds on MDL Estimators for Linear Regression Models with Application to Simple ReLU Neural Networks
Yoshinari Takeishi, Jun'ichi Takeuchi
To investigate the theoretical foundations of deep learning from the viewpoint of the minimum description length (MDL) principle, we analyse risk bounds of MDL estimators based on…
Approximate Spectral Decomposition of Fisher Information Matrix for Simple ReLU Networks
Yoshinari Takeishi, Masazumi Iida, Jun'ichi Takeuchi
We argue the Fisher information matrix (FIM) of one hidden layer networks with the ReLU activation function. For a network, let denote the weight matrix from the $…
An Improved Analysis of Least Squares Superposition Codes with Bernoulli Dictionary
Yoshinari Takeishi, Jun'ichi Takeuchi
For the additive white Gaussian noise channel with average power constraint, sparse superposition codes, proposed by Barron and Joseph in 2010, achieve the capacity. While the code…