SlimFit: Memory-Efficient Fine-Tuning of Transformer-based Models Using Training Dynamics

Arash Ardakani, Altan Haan, Shangyin Tan +4

Transformer-based models, such as BERT and ViT, have achieved state-of-the-art results across different natural language processing (NLP) and computer vision (CV) tasks. However, t…