Bayesian Mixture of Experts For Large Language Models

Maryam Dialameh, Hossein Rajabzadeh, Weiwei Zhang +2

We present Bayesian Mixture of Experts (Bayesian-MoE), a post-hoc uncertainty estimation framework for fine-tuned large language models (LLMs) based on Mixture-of-Experts architect…