Data-Efficiency with a Single GPU: An Exploration of Transfer Methods for Small Language Models

Alon Albalak, Akshat Shrivastava, Chinnadhurai Sankar +2

Multi-task learning (MTL), instruction tuning, and prompting have recently been shown to improve the generalizability of large language models to new tasks. However, the benefits o…