6 papers
Decision Aggregation under Quantal Response
Zhihuan Huang, Yichong Xia, Yuqing Kong
The effectiveness of collective decision-making is often challenged by the bounded rationality and inherent stochasticity of individual agents. We investigate this by analyzing how…
PMIScore: An Unsupervised Approach to Quantify Dialogue Engagement
Yongkang Guo, Zhihuan Huang, Yuqing Kong
High dialogue engagement is a crucial indicator of an effective conversation. A reliable measure of engagement could help benchmark large language models, enhance the effectiveness…
Jailbreaking LLMs via Calibration
Yuxuan Lu, Yongkang Guo, Yuqing Kong
Safety alignment in Large Language Models (LLMs) often creates a systematic discrepancy between a model's aligned output and the underlying pre-aligned data distribution. We propos…
Benchmarking LLMs' Judgments with No Gold Standard
Shengwei Xu, Yuxuan Lu, Grant Schoenebeck +1
We introduce the GEM (Generative Estimator for Mutual Information), an evaluation metric for assessing language generation by Large Language Models (LLMs), particularly in generati…
Eliciting Informative Text Evaluations with Large Language Models
Yuxuan Lu, Shengwei Xu, Yichi Zhang +2
Peer prediction mechanisms motivate high-quality feedback with provable guarantees. However, current methods only apply to rather simple reports, like multiple-choice or scalar num…
How Gold to Make the Golden Snitch: Designing the "Game Changer" in Esports
Zhihuan Huang, Yuxuan Lu, Yongkang Guo +1
Many battling games utilize a special item (e.g. Roshan in Defense of the Ancients 2 (DOTA 2), Baron Nashor in League of Legends (LOL), Golden Snitch in Quidditch) as a potential `…