Contrast Matters: Understanding Robustness of In-Context Fine-Tuning to Target-Context Relatedness

Deeptanshu Malu, Deevyanshu Malu, Aditya Nemiwal +1

In-context fine-tuning (IC-Train), training an LLM with labeled examples in-context, is increasingly used in place of standard fine-tuning for domain adaptation and continual absor…