One Size Does Not Fit All: Distribution-Guided Tensor-Wise Adaptive Sparsification for Model Merging

Yingfeng Luo, Dingyang Lin, Junxin Wang +10

Merging multiple task-specific fine-tuned checkpoints into a single multi-task model provides an attractive alternative to joint multi-task training. To mitigate cross-task interfe…