How fine can fine-tuning be? Learning efficient language models

Evani Radiya-Dixit, Xin Wang

State-of-the-art performance on language understanding tasks is now achieved with increasingly large networks; the current record holder has billions of parameters. Given a languag…