45 citations · 93 across the 25 of their papers we have counts for
Showing 2023Show all
2 papers · 1 filter
cs.LG2023
On the Parameterization of Second-Order Optimization Effective Towards the Infinite Width
Satoki Ishikawa, Ryo Karakida
Second-order optimization has been developed to accelerate the training of deep neural networks and it is being applied to increasingly larger-scale models. In this study, towards…
cs.LG2023★ 1 cited
Understanding MLP-Mixer as a Wide and Sparse MLP
Tomohiro Hayase, Ryo Karakida
Multi-layer perceptron (MLP) is a fundamental component of deep learning, and recent MLP-based architectures, especially the MLP-Mixer, have achieved significant empirical success.…