Adaptive Mutual Distillation for Balanced Multi-Task Post-Training of Large Language Models

Baohang Li, Xiaocheng Feng, Yichong Huang +6

Multi-task post-training of large language models (LLMs) aims to improve performance across tasks with unequal amounts of training data. Existing methods focus primarily on balanci…