Few-Shot Learning by Dimensionality Reduction in Gradient Space

Martin Gauch, Maximilian Beck, Thomas Adler +10

We introduce SubGD, a novel few-shot learning method which is based on the recent finding that stochastic gradient descent updates tend to live in a low-dimensional parameter subsp…