170 citations · 364 across the 16 of their papers we have counts for
17 papers
Leveraging Semantic Type Dependencies for Clinical Named Entity Recognition
Linh Le, Guido Zuccon, Gianluca Demartini +2
Previous work on clinical relation extraction from free-text sentences leveraged information about semantic types from clinical knowledge bases as a part of entity representations.…
Identification of Regulatory Requirements Relevant to Business Processes: A Comparative Study on Generative AI, Embedding-based Ranking, Crowd and Expert-driven Methods
Catherine Sai, Shazia Sadiq, Lei Han +2
Organizations face the challenge of ensuring compliance with an increasing amount of requirements from various regulatory documents. Which requirements are relevant depends on aspe…
Data Bias Management
Gianluca Demartini, Kevin Roitero, Stefano Mizzaro
Due to the widespread use of data-powered systems in our everyday lives, concepts like bias and fairness gained significant attention among researchers and practitioners, in both i…
On the Impact of Data Quality on Image Classification Fairness
Aki Barry, Lei Han, Gianluca Demartini
With the proliferation of algorithmic decision-making, increased scrutiny has been placed on these systems. This paper explores the relationship between the quality of the training…
Perspectives on Large Language Models for Relevance Judgment
Guglielmo Faggioli, Laura Dietz, Charles Clarke +8
When asked, large language models (LLMs) like ChatGPT claim that they can assist with relevance judgments but it is not clear whether automated judgments can reliably be used in ev…
Crowdsourced Fact-Checking at Twitter: How Does the Crowd Compare With Experts?
Mohammed Saeed, Nicolas Traub, Maelle Nicolas +2
Fact-checking is one of the effective solutions in fighting online misinformation. However, traditional fact-checking is a process requiring scarce expert human resources, and thus…
Managing Bias in Human-Annotated Data: Moving Beyond Bias Removal
Gianluca Demartini, Kevin Roitero, Stefano Mizzaro
Due to the widespread use of data-powered systems in our everyday lives, the notions of bias and fairness gained significant attention among researchers and practitioners, in both…
The Many Dimensions of Truthfulness: Crowdsourcing Misinformation Assessments on a Multidimensional Scale
Michael Soprano, Kevin Roitero, David La Barbera +4
Recent work has demonstrated the viability of using crowdsourcing as a tool for evaluating the truthfulness of public statements. Under certain conditions such as: (1) having a bal…
On the state of reporting in crowdsourcing experiments and a checklist to aid current practices
Jorge Ramírez, Burcu Sayin, Marcos Baez +4
Crowdsourcing is being increasingly adopted as a platform to run studies with human subjects. Running a crowdsourcing experiment involves several choices and strategies to successf…
Can the Crowd Judge Truthfulness? A Longitudinal Study on Recent Misinformation about COVID-19
Kevin Roitero, Michael Soprano, Beatrice Portelli +6
Recently, the misinformation problem has been addressed with a crowdsourcing-based approach: to assess the truthfulness of a statement, instead of relying on a few experts, a crowd…
The COVID-19 Infodemic: Can the Crowd Judge Recent Misinformation Objectively?
Kevin Roitero, Michael Soprano, Beatrice Portelli +5
Misinformation is an ever increasing problem that is difficult to solve for the research community and has a negative impact on the society at large. Very recently, the problem has…
Can The Crowd Identify Misinformation Objectively? The Effects of Judgment Scale and Assessor's Background
Kevin Roitero, Michael Soprano, Shaoyang Fan +3
Truthfulness judgments are a fundamental step in the process of fighting misinformation, as they are crucial to train and evaluate classifiers that automatically distinguish true a…
Proceedings of the KG-BIAS Workshop 2020 at AKBC 2020
Edgar Meij, Tara Safavi, Chenyan Xiong +3
The KG-BIAS 2020 workshop touches on biases and how they surface in knowledge graphs (KGs), biases in the source data that is used to create KGs, methods for measuring or remediati…
Non-Parametric Class Completeness Estimators for Collaborative Knowledge Graphs -- The Case of Wikidata
Michael Luggen, Djellel Difallah, Cristina Sarasua +2
Collaborative Knowledge Graph platforms allow humans and automated scripts to collaborate in creating, updating and interlinking entities and facts. To ensure both the completeness…
FashionBrain Project: A Vision for Understanding Europe's Fashion Data Universe
Alessandro Checco, Gianluca Demartini, Alexander Loeser +7
A core business in the fashion industry is the understanding and prediction of customer needs and trends. Search engines and social networks are at the same time a fundamental brid…
The Effect of Class Imbalance and Order on Crowdsourced Relevance Judgments
Rehab K. Qarout, Alessandro Checco, Gianluca Demartini
In this paper we study the effect on crowd worker efficiency and effectiveness of the dominance of one class in the data they process. We aim at understanding if there is any posit…
Pairwise, Magnitude, or Stars: What's the Best Way for Crowds to Rate?
Alessandro Checco, Gianluca Demartini
We compare three popular techniques of rating content: the ubiquitous five star rating, the less used pairwise comparison, and the recently introduced (in crowdsourcing) magnitude…