2 citations · 3 across the 3 of their papers we have counts for
3 papers · 1 filter
Measuring the local shape of learning-rate transfer curves
Satoki Ishikawa, Rio Yokota, Ryo Karakida
Understanding the learning-rate (LR) dependence of trained model performance is essential for better optimization of neural networks, but the appropriate size of the LR depends on…
Lion Cub: Minimizing Communication Overhead in Distributed Lion
Satoki Ishikawa, Tal Ben-Nun, Brian Van Essen +2
Communication overhead is a key challenge in distributed deep learning, especially on slower Ethernet interconnects, and given current hardware trends, communication is likely to b…
Local Loss Optimization in the Infinite Width: Stable Parameterization of Predictive Coding Networks and Target Propagation
Satoki Ishikawa, Rio Yokota, Ryo Karakida
Local learning, which trains a network through layer-wise local targets and losses, has been studied as an alternative to backpropagation (BP) in neural computation. However, its a…