The Costly Dilemma: Generalization, Evaluation and Cost-Optimal Deployment of Large Language Models

Abi Aryan, Aakash Kumar Nain, Andrew McMahon +2

When deploying machine learning models in production for any product/application, there are three properties that are commonly desired. First, the models should be generalizable, i…