On the Effect of Instruction Tuning Loss on Generalization

Anwoy Chatterjee, H S V N S Kowndinya Renduchintala, Sumit Bhatia +1

Instruction Tuning has emerged as a pivotal post-training paradigm that enables pre-trained language models to better follow user instructions. Despite its significance, little att…