Knowledge Composition using Task Vectors with Learned Anisotropic Scaling

Frederic Z. Zhang, Paul Albert, Cristian Rodriguez-Opazo +2

Pre-trained models produce strong generic representations that can be adapted via fine-tuning. The learned weight difference relative to the pre-trained model, known as a task vect…