TOAST: Transfer Learning via Attention Steering

Baifeng Shi, Siyu Gai, Trevor Darrell +1

Transfer learning involves adapting a pre-trained model to novel downstream tasks. However, we observe that current transfer learning methods often fail to focus on task-relevant f…