Accurate Retraining-free Pruning for Pretrained Encoder-based Language Models

Seungcheol Park, Hojun Choi, U Kang

Given a pretrained encoder-based language model, how can we accurately compress it without retraining? Retraining-free structured pruning algorithms are crucial in pretrained langu…