Rethinking LLM Ensembling from the Perspective of Mixture Models

Jiale Fu, Yuchu Jiang, Peijun Wu +3

Model ensembling is a well-established technique for improving the performance of machine learning models. Conventionally, this involves averaging the output distributions of multi…