124 citations · 666 across the 43 of their papers we have counts for
5 papers · 1 filter
ASDL: A Unified Interface for Gradient Preconditioning in PyTorch
Kazuki Osawa, Satoki Ishikawa, Rio Yokota +2
Gradient preconditioning is a key technique to integrate the second-order information into gradients for improving and extending gradient-based learning algorithms. In deep learnin…
Empirical Study on Optimizer Selection for Out-of-Distribution Generalization
Hiroki Naganuma, Kartik Ahuja, Shiro Takagi +5
Modern deep learning systems do not generalize well when the test data distribution is slightly different to the training data distribution. While much promising work has been acco…
OPIRL: Sample Efficient Off-Policy Inverse Reinforcement Learning via Distribution Matching
Hana Hoshino, Kei Ota, Asako Kanezaki +1
Inverse Reinforcement Learning (IRL) is attractive in scenarios where reward engineering can be tedious. However, prior IRL algorithms use on-policy transitions, which require inte…
Scalable and Practical Natural Gradient for Large-Scale Deep Learning
Kazuki Osawa, Yohei Tsuji, Yuichiro Ueno +3
Large-scale distributed training of deep neural networks results in models with worse generalization performance as a result of the increase in the effective mini-batch size. Previ…
Large-Scale Distributed Second-Order Optimization Using Kronecker-Factored Approximate Curvature for Deep Convolutional Neural Networks
Kazuki Osawa, Yohei Tsuji, Yuichiro Ueno +3
Large-scale distributed training of deep neural networks suffer from the generalization gap caused by the increase in the effective mini-batch size. Previous approaches try to solv…