NESSiE: The Necessary Safety Benchmark -- Identifying Errors that should not Exist

Johannes Bertram, Jonas Geiping

We introduce NESSiE, the NEceSsary SafEty benchmark for large language models (LLMs). With minimal test cases of information and access security, NESSiE reveals safety-relevant fai…