35 citations · 84 across the 8 of their papers we have counts for
9 papers
Knowledge Distillation under Teacher Misspecification: An Order-Parameter Analysis of the Gap between Teacher Mimicry and Task Performance
Kazuyuki Hara, Hideitsu Hino
Knowledge distillation trains a small student model to reproduce the outputs of a large teacher model, and its progress is typically monitored through the teacher--student discrepa…
Analysis of Dropout in Online Learning
Kazuyuki Hara
Deep learning is the state-of-the-art in fields such as visual object recognition and speech recognition. This learning uses a large number of layers and a huge number of units and…
Statistical Mechanics of Node-perturbation Learning with Noisy Baseline
Kazuyuki Hara, Kentaro Katahira, Masato Okada
Node-perturbation learning is a type of statistical gradient descent algorithm that can be applied to problems where the objective function is not explicitly formulated, including…
Analysis of dropout learning regarded as ensemble learning
Kazuyuki Hara, Daisuke Saitoh, Hayaru Shouno
Deep learning is the state-of-the-art in fields such as visual object recognition and speech recognition. This learning uses a large number of layers, huge number of units, and con…
Statistical Mechanics of On-line Ensemble Teacher Learning through a Novel Perceptron Learning Rule
Kazuyuki Hara, Seiji Miyoshi
In ensemble teacher learning, ensemble teachers have only uncertain information about the true teacher, and this information is given by an ensemble consisting of an infinite numbe…
Optimization of the Asymptotic Property of Mutual Learning Involving an Integration Mechanism of Ensemble Learning
Kazuyuki Hara, Takahiro Yamada
We propose an optimization method of mutual learning which converges into the identical state of optimum ensemble learning within the framework of on-line learning, and have analyz…