Showing cs.LGShow all
2 papers · 1 filter
cs.LG2025
Expert Merging in Sparse Mixture of Experts with Nash Bargaining
Dung V. Nguyen, Anh T. Nguyen, Minh H. Nguyen +6
Existing expert merging strategies for Sparse Mixture of Experts (SMoE) typically rely on input-dependent or input-independent averaging of expert parameters, but often lack a prin…
cs.LG2025
CAMEx: Curvature-aware Merging of Experts
Dung V. Nguyen, Minh H. Nguyen, Luc Q. Nguyen +3
Existing methods for merging experts during model training and fine-tuning predominantly rely on Euclidean geometry, which assumes a flat parameter space. This assumption can limit…