Augmenting Human Evaluation with LLM Judges: How Many Human Reviews Do You Need?

Jane Paik Kim

Large language models (LLMs) are increasingly used as automated evaluators of AI systems, including in high-stakes applications. In this role, LLMs are used to generate judgments a…