Beyond Full-Model Rollback: AuroSFT for Adapter-State Multi-Task Fine-Tuning

Yue Han, Ziniu Liu

Multi-task supervised fine-tuning (SFT) often casts a heterogeneous data mixture as a single optimization problem, even though different tasks may reach their best generalization a…