58 citations · 134 across the 8 of their papers we have counts for
9 papers · 1 filter
The Effect of Document Summarization on LLM-Based Relevance Judgments
Samaneh Mohtadi, Kevin Roitero, Stefano Mizzaro +1
Relevance judgments are central to the evaluation of Information Retrieval (IR) systems, but obtaining them from human annotators is costly and time-consuming. Large Language Model…
Efficiency and Effectiveness of LLM-Based Summarization of Evidence in Crowdsourced Fact-Checking
Kevin Roitero, Dustin Wright, Michael Soprano +2
Evaluating the truthfulness of online content is critical for combating misinformation. This study examines the efficiency and effectiveness of crowdsourced truthfulness assessment…
Managing Bias in Human-Annotated Data: Moving Beyond Bias Removal
Gianluca Demartini, Kevin Roitero, Stefano Mizzaro
Due to the widespread use of data-powered systems in our everyday lives, the notions of bias and fairness gained significant attention among researchers and practitioners, in both…
The Many Dimensions of Truthfulness: Crowdsourcing Misinformation Assessments on a Multidimensional Scale
Michael Soprano, Kevin Roitero, David La Barbera +4
Recent work has demonstrated the viability of using crowdsourcing as a tool for evaluating the truthfulness of public statements. Under certain conditions such as: (1) having a bal…
Can the Crowd Judge Truthfulness? A Longitudinal Study on Recent Misinformation about COVID-19
Kevin Roitero, Michael Soprano, Beatrice Portelli +6
Recently, the misinformation problem has been addressed with a crowdsourcing-based approach: to assess the truthfulness of a statement, instead of relying on a few experts, a crowd…
The COVID-19 Infodemic: Can the Crowd Judge Recent Misinformation Objectively?
Kevin Roitero, Michael Soprano, Beatrice Portelli +5
Misinformation is an ever increasing problem that is difficult to solve for the research community and has a negative impact on the society at large. Very recently, the problem has…