4 papers
The Wrong Kind of Right: Quantifying and Localizing Misfired Alignment in LLMs
Naihao Deng, Yiming Feng, Chimaobi Okite +4
Warning: This paper studies stereotypes and biases, and contains potentially disturbing examples, used for illustration purposes only. Our findings should not be interpreted as an…
Unstructured Evidence Attribution for Long Context Query Focused Summarization
Dustin Wright, Zain Muhammad Mujahid, Lu Wang +2
Large language models (LLMs) are capable of generating coherent summaries from very long contexts given a user query, and extracting and citing evidence spans helps improve the tru…
Structured Moral Reasoning in Language Models: A Value-Grounded Evaluation Framework
Mohna Chakraborty, Lu Wang, David Jurgens
Large language models (LLMs) are increasingly deployed in domains requiring moral understanding, yet their reasoning often remains shallow, and misaligned with human reasoning. Unl…
Evaluation Framework for AI Systems in "the Wild"
Sarah Jabbour, Trenton Chang, Anindya Das Antar +13
Generative AI (GenAI) models have become vital across industries, yet current evaluation methods have not adapted to their widespread use. Traditional evaluations often rely on ben…