Cross-Dataset Design Discussion Mining
arXiv:2001.01424 · doi:10.1109/SANER48275.2020.9054792
Abstract
Being able to identify software discussions that are primarily about design, which we call design mining, can improve documentation and maintenance of software systems. Existing design mining approaches have good classification performance using natural language processing (NLP) techniques, but the conclusion stability of these approaches is generally poor. A classifier trained on a given dataset of software projects has so far not worked well on different artifacts or different datasets. In this study, we replicate and synthesize these earlier results in a meta-analysis. We then apply recent work in transfer learning for NLP to the problem of design mining. However, for our datasets, these deep transfer learning classifiers perform no better than less complex classifiers. We conclude by discussing some reasons behind the transfer learning approach to design mining.
arXiv commentaccepted for SANER 2020, Feb, London, ON. 12 pages. Replication package: https://doi.org/10.5281/zenodo.3590126
References in corpus (13)
- Scikit-learn: Machine Learning in Python
- SMOTE: Synthetic Minority Over-sampling Technique
- Efficient Estimation of Word Representations in Vector Space
- Distributed Representations of Sentences and Documents
- Enriching Word Vectors with Subword Information
- SOTorrent: Reconstructing and Analyzing the Evolution of Stack Overflow Posts
- Evolution of statistical analysis in empirical software engineering research: Current state and steps forward
- Can We Use SE-specific Sentiment Analysis Tools in a Cross-Platform Setting?
- Hyperparameter Optimization for Effort Estimation
- A systematic mapping study on cross-project defect prediction
- Replication studies considered harmful
- Cross-Dataset Design Discussion Mining
- Rationale in Development Chat Messages: An Exploratory Study