4 papers · 1 filter
Measuring Judgment Quality in Natural-Language Explanations: Evidence from Forecasting Tournaments
Christopher W. Karvetski, Sheldon S. Huang, Simas Kučinskas +4
Decision-makers routinely rely on expert judgments accompanied by written explanations, yet explanation quality is difficult to measure at scale. Forecasting tournaments offer a na…
What a One-Hour Interview Does Not Tell an LLM About You: Interviews Elicit Traits, Not Decision Policies
Ruoxi Su, Yuhan Liu, Jingyu Hu
Large language models are increasingly asked to stand in for particular people, and interviewing the person is the natural way to tell a model who they are. We test what a one-hour…
StakeBench: Evaluating Language Understanding Grounded in Market Commitment
Yunhua Pei, Jingyu Hu, Yiwei Shi +3
Existing financial NLP benchmarks often rely on labels supplied by outside observers, measuring how language is perceived rather than what speakers have committed to in the market.…
Fine-Grained Interpretation of Political Opinions in Large Language Models
Jingyu Hu, Mengyue Yang, Mengnan Du +1
Studies of LLMs' political opinions mainly rely on evaluations of their open-ended responses. Recent work indicates that there is a misalignment between LLMs' responses and their i…