Training an LLM-as-a-Judge Model: Pipeline, Insights, and Practical Lessons

Renjun Hu, Yi Cheng, Libin Meng +4

The rapid advancement of large language models (LLMs) has opened new possibilities for their adoption as evaluative judges. This paper introduces Themis, a fine-tuned LLM judge tha…