9 papers
Disagreeing Rationales: Rethinking Classification and Explainability Evaluation in Hate Speech Detection
Benedetta Muscato, Beiduo Chen, Gizem Gezici +2
Human disagreement is ubiquitous and well-known in labeling. However, variation in explanations, captured through token-level human rationales, remains far less explored. At the sa…
Reasoning that Travels: Dissecting How Chain-of-Thought Transfers Across Models
Xinyuan Cheng, Beiduo Chen, Philipp Mondorf +1
Large reasoning models (LRMs) often generate extensive chain-of-thought (CoT) traces before producing a final answer. As explicit textual artifacts, these traces can be passed to o…
Human Label Variation as Stable Signal: Learning Annotator-Specific Explanation Behavior via Cross-Annotator Preference Optimization
Beiduo Chen, Pingjun Hong, Ziyun Zhang +3
Free-text explanations extend human label variation (HLV) beyond label disagreement by revealing the reasoning and preferences behind annotators' decisions. We study whether large…
Decoupling the Effect of Chain-of-Thought Reasoning: A Human Label Variation Perspective
Beiduo Chen, Tiancheng Hu, Caiqi Zhang +3
Reasoning-tuned LLMs utilizing long Chain-of-Thought (CoT) excel at single-answer tasks, yet their ability to model Human Label Variation--which requires capturing probabilistic am…
Agree, Disagree, Explain: Decomposing Human Label Variation in NLI through the Lens of Explanations
Pingjun Hong, Beiduo Chen, Siyao Peng +3
Natural Language Inference (NLI) datasets often exhibit human label variation. To better understand these variations, explanation-based approaches analyze the underlying reasoning…
LiTEx: A Linguistic Taxonomy of Explanations for Understanding Within-Label Variation in Natural Language Inference
Pingjun Hong, Beiduo Chen, Siyao Peng +2
There is increasing evidence of Human Label Variation (HLV) in Natural Language Inference (NLI), where annotators assign different labels to the same premise-hypothesis pair. Howev…