Peer review vs bibliometrics: which method better predicts the scholarly impact of publications?
arXiv:1909.06209 · doi:10.1007/s11192-019-03184-y
Abstract
In this work, we try to answer the question of which method, peer review vs bibliometrics, better predicts the future overall scholarly impact of scientific publications. We measure the agreement between peer review evaluations of Web of Science indexed publications submitted to the first Italian research assessment exercise and long-term citations of the same publications. We do the same for an early citation-based indicator. We find that the latter shows stronger predictive power, i.e., it more reliably predicts late citations in all the disciplinary areas examined, and for any citation time window starting one year after publication.
References in corpus (10)
- Defining and identifying Sleeping Beauties in science
- Predicting long-term publication impact through a combination of early citations and journal impact factor
- Assessing the varying level of impact measurement accuracy as a function of the citation window length
- National research assessment exercises: a comparison of peer review and bibliometrics rankings
- Revisiting the scientometric conceptualization of impact and its measurement
- Systematic analysis of agreement between metrics and peer review in the UK REF
- The dispersion of research performance within and between universities as a potential indicator of the competitive intensity in higher education systems
- National peer-review research assessment exercises for the hard sciences can be a complete waste of money: the Italian case
- Refrain from adopting the combination of citation and journal metrics to grade publications, as used in the Italian national research assessment exercise (VQR 2011-2014)
- Do Citations and Readership Identify Seminal Publications?