Soft-TransFormers for Continual Learning

Haeyong Kang, Chang D. Yoo

Inspired by the Well-initialized Lottery Ticket Hypothesis (WLTH), we introduce Soft-TransFormers (Soft-TF), a continual learning framework that adapts a frozen pre-trained Transfo…