MeRoTune: RoPE-Safe Merging with a Tunable Dial

Salman Faroz

When you merge two fine-tuned models from the same base checkpoint by simply averaging their weights, you implicitly assume their attention subspaces are still aligned. Recent work…