6 papers
SAGE: Schema-Guided LLMs for Grant Review
Erik Varapaev, Andrei Chetvergov, Stepan Ukolov +3
Grant reviewers must apply detailed criteria to application forms, budgets, and supporting documents while producing assessments that colleagues can inspect. We present SAGE, Schem…
PROOF: Profiling Reliability of Object-Level Facts in Large Language Models
Andrei Chetvergov, Mikhail Solovev, Timofei Sivoraksha +4
Aggregate factuality scores hide where a language model succeeds, which relations it confuses, and whether an answer survives innocuous changes to the question or decoder. We intro…
STONIC: A Layered Measurement Contract for LLM Value Profiling
Andrei Chetvergov, Stepan Ukolov, Timofei Sivoraksha +4
LLM value studies often merge questionnaire ratings, pairwise choices, and values inferred from generated text into one profile. That merge assumes that the three observations desc…
VIBE: A VAD-Informed Benchmark for Entity-Centered Affective Profiling of Large Language Model Outputs
Andrei Chetvergov, Alexander Evseev, Timofei Sivoraksha +4
Large language models routinely describe socially salient targets, including political figures, countries, religions, organizations, historical events, and social groups, encoding…
REGARD: Regional Affective Differences in Large Language Models
Andrei Chetvergov, Alexander Evseev, Mikhail Solovev +5
Large language models trained and aligned within different linguistic and regional ecosystems may frame the same political, cultural, and geopolitical entities in different ways. S…
Which Values Do LLMs Confuse? A Schwartz-Based Recognition Study
Andrei Chetvergov, Stepan Ukolov, Timofei Sivoraksha +5
Large language models are increasingly evaluated through the values they endorse, but such evaluations presuppose that models can identify the value expressed in a concrete situati…