On the Utility and Factual Reliability of Pruned Mixture-of-Experts Models in the Biomedical Domain

Atsuki Yamaguchi, Szymon Palucha, Léo Bijar +2

Mixture-of-Experts (MoE) models offer inference speedups via selective activation but impose substantial memory requirements because the whole network must remain loaded. Structure…