What Happens When Small Is Made Smaller? Exploring the Impact of Compression on Small Data Pretrained Language Models

Busayo Awobade, Mardiyyah Oduwole, Steven Kolawole

Compression techniques have been crucial in advancing machine learning by enabling efficient training and deployment of large-scale language models. However, these techniques have…