Ecosystem-wide influences on pull request decisions: insights from NPM

Abstract

The pull-based development model facilitates global collaboration within open-source software projects. However, whereas it is increasingly common for software to depend on other projects in their ecosystem, most research on the pull request decision-making process explored factors within projects, not the broader software ecosystem they comprise. We uncover ecosystem-wide factors that influence pull request acceptance decisions. We collected a dataset of approximately 1.8 million pull requests and 2.1 million issues from 20,052 GitHub projects within the NPM ecosystem. Of these, 98% depend on another project in the dataset, enabling studying collaboration across dependent projects. We employed social network analysis to create a collaboration network in the ecosystem, and mixed effects logistic regression and random forest techniques to measure the impact and predictive strength of the tested features. We find that gaining experience within the software ecosystem through active participation in issue-tracking systems, submitting pull requests, and collaborating with pull request integrators and experienced developers benefits all open-source contributors, especially project newcomers. These results are complemented with an exploratory qualitative analysis of 538 pull requests. We find that developers with ecosystem experience make different contributions than users without. Zooming in on a subset of 111 pull requests with clear ecosystem involvement, we find 3 overarching and 10 specific reasons why developers involve ecosystem projects in their pull requests. The results show that combining ecosystem-wide factors with features studied in previous work to predict the outcome of pull requests reached an overall F1 score of 0.92. However, the outcomes of pull requests submitted by newcomers are harder to predict.

52 pages, 3 figures, 7 tables, 1 appendix. The abstract in the arXiv metadata is shortened due to size constraints

References in corpus (11)

• Emergence of scaling in random networks
• Multilayer Networks
• A ground-truth dataset and classification model for detecting bots in GitHub issue and PR comments
• Analysis of Neighbourhoods in Multi-layered Dynamic Social Networks
• Detecting and Characterizing Bots that Commit Code
• Modern Code Reviews -- Survey of Literature and Practice
• Generative AI for Pull Request Descriptions: Adoption, Impact, and Developer Interventions
• Effect of Technical and Social Factors on Pull Request Quality for the NPM Ecosystem
• Pots of Gold at the End of the Rainbow: What is Success for Open Source Contributors?
• Bot Detection in GitHub Repositories
• How does Software Change?