When Ensembling Smaller Models is More Efficient than Single Large Models

Dan Kondratyuk, Mingxing Tan, Matthew Brown +1

Ensembling is a simple and popular technique for boosting evaluation performance by training multiple models (e.g., with different initializations) and aggregating their prediction…