Crowd-OM: Crowdsourcing for Ontology Matching Validation
arXiv:2605.12226
Abstract
Recent advances in large language models (LLMs) pose new challenges for ontology matching (OM). While OM systems built on LLMs have shown remarkable capabilities in discovering more matching candidates, traditional OM validation that relies on domain experts has become overwhelming. Although crowdsourcing can expose OM validation to large online communities, bias and errors from diverse annotators can reduce the quality of crowdsourcing for OM validation. In this study, we explore the use of crowdsourcing for OM validation and introduce a novel quality assurance crowdsourcing system called Crowd-OM. We propose three domain-specific mechanisms (differential trustworthiness, coherence pre-filling, and time-dependent opinion) to ensure the quality of crowdsourcing for OM validation. Crowd-OM can be integrated with existing OM systems to enable human-in-the-loop validation. The evaluation shows its effectiveness in handling various annotators and different annotation scenarios. We discuss two real-world use cases and current limitations for improvement.
6 pages, 7 figures