Developing and Maintaining an Open-Source Repository of AI Evaluations: Challenges and Insights

Alexandra Abbas, Celia Waggoner, Justin Olive

AI evaluations have become critical tools for assessing large language model capabilities and safety. This paper presents practical insights from eight months of maintaining $inspe…