29 citations · 36 across the 21 of their papers we have counts for
4 papers · 1 filter
Fixed-Weight Difference Target Propagation
Tatsukichi Shibuya, Nakamasa Inoue, Rei Kawakami +1
Target Propagation (TP) is a biologically more plausible algorithm than the error backpropagation (BP) to train deep networks, and improving practicality of TP is an open issue. TP…
Parameter Efficient Transfer Learning for Various Speech Processing Tasks
Shinta Otake, Rei Kawakami, Nakamasa Inoue
Fine-tuning of self-supervised models is a powerful transfer learning method in a variety of fields, including speech processing, since it can utilize generic feature representatio…
PoF: Post-Training of Feature Extractor for Improving Generalization
Ikuro Sato, Ryota Yamada, Masayuki Tanaka +2
It has been intensively investigated that the local shape, especially flatness, of the loss landscape near a minimum plays an important role for generalization of deep models. We d…
Pre-training Vision Transformers with Formula-driven Supervised Learning
Hirokatsu Kataoka, Sora Takashima, Ryo Hayamizu +6
In the present work, we show that the performance of formula-driven supervised learning (FDSL) can match or even exceed that of ImageNet-21k and can approach that of the JFT-300M d…