Modular Norm RandOpt: Population-Efficient Ensembling through Architecture-Aware Perturbations

Kirato Yoshihara, Hiroaki Hamade

RandOpt samples weight-perturbed language models and ensembles top-ranked candidates through plurality voting, but its global perturbation scale ignores heterogeneous module geomet…