On Surgical Fine-tuning for Language Encoders

Abhilasha Lodha, Gayatri Belapurkar, Saloni Chalkapurkar +5

Fine-tuning all the layers of a pre-trained neural language encoder (either using all the parameters or using parameter-efficient methods) is often the de-facto way of adapting it…