Less is More: Selective Layer Finetuning with SubTuning

Gal Kaplun, Andrey Gurevich, Tal Swisa +3

Finetuning a pretrained model has become a standard approach for training neural networks on novel tasks, resulting in fast convergence and improved performance. In this work, we s…