6 papers
Reference-Free Evaluation of Reasoning in Open-Ended Question Answering
Guneet Singh Kohli, Yuxiang Zhou, Michael Sejr Schlichtkrull +2
AI-generated answers in high-stakes domains are often fluent but difficult to verify, especially when they contain multi-step reasoning rather than a single final answer. We propos…
MEDIAREF: A Public Knowledge Store for Media Background Checks
Benjamin Nichols, Michael Schlichtkrull, Nedjma Ousidhoum
LLM-based retrieval-augmented generation (RAG) is increasingly used for automated fact-checking (AFC) and related tasks. By grounding LLM outputs in retrieved evidence, RAG-based s…
AVerImaTeC: A Dataset for Automatic Verification of Image-Text Claims with Evidence from the Web
Rui Cao, Zifeng Ding, Zhijiang Guo +2
Textual claims are often accompanied by images to enhance their credibility and spread on social media, but this also raises concerns about the spread of misinformation. Existing d…
Social Good or Scientific Curiosity? Uncovering the Research Framing Behind NLP Artefacts
Eric Chamoun, Nedjma Ousidhoum, Michael Schlichtkrull +1
Clarifying the research framing of NLP artefacts (e.g., models, datasets, etc.) is crucial to aligning research with practical applications. Recent studies manually analyzed NLP re…
Ev2R: Evaluating Evidence Retrieval in Automated Fact-Checking
Mubashara Akhtar, Michael Schlichtkrull, Andreas Vlachos
Current automated fact-checking (AFC) approaches typically evaluate evidence either implicitly via the predicted verdicts or through exact matches with predefined closed knowledge…
IYKYK: Using language models to decode extremist cryptolects
Christine de Kock, Arij Riabi, Zeerak Talat +3
Extremist groups develop complex in-group language, also referred to as cryptolects, to exclude or mislead outsiders. We investigate the ability of current language technologies to…