Continual Learning Under Language Shift

Evangelia Gogoulou, Timothée Lesort, Magnus Boman +1

The recent increase in data and model scale for language model pre-training has led to huge training costs. In scenarios where new data become available over time, updating a model…