How Data Scientists Review the Scholarly Literature
arXiv:2301.03774 · doi:10.1145/3576840.3578309
Abstract
Keeping up with the research literature plays an important role in the workflow of scientists - allowing them to understand a field, formulate the problems they focus on, and develop the solutions that they contribute, which in turn shape the nature of the discipline. In this paper, we examine the literature review practices of data scientists. Data science represents a field seeing an exponential rise in papers, and increasingly drawing on and being applied in numerous diverse disciplines. Recent efforts have seen the development of several tools intended to help data scientists cope with a deluge of research and coordinated efforts to develop AI tools intended to uncover the research frontier. Despite these trends indicative of the information overload faced by data scientists, no prior work has examined the specific practices and challenges faced by these scientists in an interdisciplinary field with evolving scholarly norms. In this paper, we close this gap through a set of semi-structured interviews and think-aloud protocols of industry and academic data scientists (N = 20). Our results while corroborating other knowledge workers' practices uncover several novel findings: individuals (1) are challenged in seeking and sensemaking of papers beyond their disciplinary bubbles, (2) struggle to understand papers in the face of missing details and mathematical content, (3) grapple with the deluge by leveraging the knowledge context in code, blogs, and talks, and (4) lean on their peers online and in-person. Furthermore, we outline future directions likely to help data scientists cope with the burgeoning research literature.
arXiv commentCHIIR 2023 camera-ready
References in corpus (13)
- Model Cards for Model Reporting
- How Work From Home Affects Collaboration: A Large-Scale Study of Information Workers in a Natural Experiment During COVID-19
- Improving fairness in machine learning systems: What do industry practitioners need?
- Explainable Recommendation: A Survey and New Perspectives
- Searching to Learn with Instructional Scaffolding
- Threddy: An Interactive System for Personalized Thread-based Exploration and Organization of Scientific Literature
- The Disagreement Problem in Explainable Machine Learning: A Practitioner's Perspective
- Analysing Mixed Initiatives and Search Strategies during Conversational Search
- From Who You Know to What You Read: Augmenting Scientific Recommendations with Implicit Social Networks
- ConceptScope: Organizing and Visualizing Knowledge in Documents based on Domain Ontology
- Paper Plain: Making Medical Research Papers Approachable to Healthcare Consumers with Natural Language Processing
- Augmenting Scientific Creativity with Retrieval across Knowledge Domains
- POINTREC: A Test Collection for Narrative-driven Point of Interest Recommendation