CoSINT: Designing a Collaborative Capture the Flag Competition to Investigate Misinformation

Abstract

Crowdsourced investigations shore up democratic institutions by debunking misinformation and uncovering human rights abuses. However, current crowdsourcing approaches rely on simplistic collaborative or competitive models and lack technological support, limiting their collective impact. Prior research has shown that blending elements of competition and collaboration can lead to greater performance and creativity, but crowdsourced investigations pose unique analytical and ethical challenges. In this paper, we employed a four-month-long Research through Design process to design and evaluate a novel interaction style called collaborative capture the flag competitions (CoCTFs). We instantiated this interaction style through CoSINT, a platform that enables a trained crowd to work with professional investigators to identify and investigate social media misinformation. Our mixed-methods evaluation showed that CoSINT leverages the complementary strengths of competition and collaboration, allowing a crowd to quickly identify and debunk misinformation. We also highlight tensions between competition versus collaboration and discuss implications for the design of crowdsourced investigations.

arXiv commentTo appear in ACM Designing Interactive Systems 2023 (DIS 2023). To cite this paper please use the official citation available here: https://doi.org/10.1145/3563657.3595997

References in corpus (7)

• Hoaxy: A Platform for Tracking Online Misinformation
• Crowdsourced Fact-Checking at Twitter: How Does the Crowd Compare With Experts?
• Effects of Sensemaking Translucence on Distributed Collaborative Analysis
• It's Time to Do Something: Mitigating the Negative Impacts of Computing Through a Change to the Peer Review Process
• Birdwatch: Crowd Wisdom and Bridging Algorithms can Inform Understanding and Reduce the Spread of Misinformation
• Leveraging Large Language Model and Story-Based Gamification in Intelligent Tutoring System to Scaffold Introductory Programming Courses: A Design-Based Research Study
• Sedition Hunters: A Quantitative Study of the Crowdsourced Investigation into the 2021 U.S. Capitol Attack

Cited by in corpus (3)

• OSINT Research Studios: A Flexible Crowdsourcing Framework to Scale Up Open Source Intelligence Investigations
• OSINT Clinic: Co-designing AI-Augmented Collaborative OSINT Investigations for Vulnerability Assessment
• Exploring Multidimensional Checkworthiness: Designing AI-assisted Claim Prioritization for Human Fact-checkers