11 papers · 1 filter
Demystifying Multi-Agent Debate: The Role of Confidence and Diversity
Xiaochen Zhu, Caiqi Zhang, Yizhou Chi +3
Multi-agent debate (MAD) is widely used to improve large language model (LLM) performance through test-time scaling, yet recent work shows that vanilla MAD often underperforms simp…
The Role of Ambiguity in Error Prediction via Uncertainty Quantification
Ieva Raminta StaliÅ«naitÄ, James Bishop, Andreas Vlachos
The task of Error Prediction, namely predicting whether a model output is correct, is commonly tackled with Uncertainty Quantification (UQ). However, while uncertainty metrics capt…
LoVeC: Reinforcement Learning for Better Verbalized Confidence in Long-Form Generations
Caiqi Zhang, Xiaochen Zhu, Chengzu Li +2
Hallucination remains a major challenge for the safe and trustworthy deployment of large language models (LLMs) in factual content generation. Prior work has explored confidence es…
Collaborative Evaluation of Deepfake Text with Deliberation-Enhancing Dialogue Systems
Jooyoung Lee, Xiaochen Zhu, Georgi Karadzhov +3
The proliferation of generative models has presented significant challenges in distinguishing authentic human-authored content from deepfake content. Collaborative human efforts, a…
Uncertainty Quantification for Evaluating Machine Translation Bias
Ieva Raminta StaliÅ«naitÄ, Julius Cheng, Andreas Vlachos
The predictive uncertainty of machine translation (MT) models is typically used as a quality estimation proxy. In this work, we posit that apart from confidently translating when a…
Causal Estimation of Tokenisation Bias
Pietro Lesci, Clara Meister, Thomas Hofmann +2
Modern language models are typically trained over subword sequences, but ultimately define probabilities over character-strings. Ideally, the choice of the tokeniser -- which maps…