4 papers
ROK-FORTRESS: Measuring the Effect of Geopolitical Transcreation for National Security and Public Safety
Michael S. Lee, Yash Maurya, Drew Rein +13
Safety evaluations for large language models (LLMs) increasingly target high-stakes National Security and Public Safety (NSPS) risks, yet multilingual safety is mostly assessed thr…
PANORAMA: A Dataset and Benchmarks Capturing Decision Trails and Rationales in Patent Examination
Hyunseung Lim, Sooyohn Nam, Sungmin Na +7
Patent examination remains an ongoing challenge in the NLP literature even after the advent of large language models (LLMs), as it requires an extensive yet nuanced human judgment…
Mind the Blind Spots: A Focus-Level Evaluation Framework for LLM Reviews
Hyungyu Shin, Jingyu Tang, Yoonjoo Lee +6
Peer review underpins scientific progress, but it is increasingly strained by reviewer shortages and growing workloads. Large Language Models (LLMs) can automatically draft reviews…
The BiGGen Bench: A Principled Benchmark for Fine-grained Evaluation of Language Models with Language Models
Seungone Kim, Juyoung Suk, Ji Yong Cho +29
As language models (LMs) become capable of handling a wide range of tasks, their evaluation is becoming as challenging as their development. Most generation benchmarks currently as…