Training Data Selection with Gradient Orthogonality for Efficient Domain Adaptation

Xiyang Zhang, Yuanhe Tian, Hongzhi Wang +1

Fine-tuning large language models (LLMs) for specialized domains often necessitates a trade-off between acquiring domain expertise and retaining general reasoning capabilities, a p…