25 citations · 139 across the 31 of their papers we have counts for
4 papers · 1 filter
Statistical Mechanical Analysis of Catastrophic Forgetting in Continual Learning with Teacher and Student Networks
Haruka Asanuma, Shiro Takagi, Yoshihiro Nagano +3
When a computational system continuously learns from an ever-changing environment, it rapidly forgets its past experiences. This phenomenon is called catastrophic forgetting. While…
Data-Dependence of Plateau Phenomenon in Learning with Neural Network --- Statistical Mechanical Analysis
Yuki Yoshida, Masato Okada
The plateau phenomenon, wherein the loss value stops decreasing during the process of learning, has been reported by various researchers. The phenomenon is actively inspected in th…
Concept Formation and Dynamics of Repeated Inference in Deep Generative Models
Yoshihiro Nagano, Ryo Karakida, Masato Okada
Deep generative models are reported to be useful in broad applications including image generation. Repeated inference between data space and latent space in these models can denois…
Statistical Mechanics of Node-perturbation Learning with Noisy Baseline
Kazuyuki Hara, Kentaro Katahira, Masato Okada
Node-perturbation learning is a type of statistical gradient descent algorithm that can be applied to problems where the objective function is not explicitly formulated, including…