National peer-review research assessment exercises for the hard sciences can be a complete waste of money: the Italian case
arXiv:1810.13274 · doi:10.1007/s11192-012-0875-6
Abstract
There has been ample demonstration that bibliometrics is superior to peer-review for national research assessment exercises in the hard sciences. In this paper we examine the Italian case, taking the 2001-2003 university performance rankings list based on bibliometrics as benchmark. We compare the accuracy of the first national evaluation exercise, conducted entirely by peer-review, to other rankings lists prepared at zero cost, based on indicators indirectly linked to performance or available on the Internet. The results show that, for the hard sciences, the costs of conducting the Italian evaluation of research institutions could have been completely avoided.
References in corpus (3)
Cited by in corpus (12)
- Regression for citation data: An evaluation of different methods
- The VQR, Italy's second national research assessment: Methodological failures and ranking distortions
- Reviewers' ratings and bibliometric indicators: hand in hand when assessing over research proposals?
- Peer review vs bibliometrics: which method better predicts the scholarly impact of publications?
- Refrain from adopting the combination of citation and journal metrics to grade publications, as used in the Italian national research assessment exercise (VQR 2011-2014)
- The influence of time and discipline on the magnitude of correlations between citation counts and quality scores
- Measuring Metrics - A forty year longitudinal cross-validation of citations, downloads, and peer review in Astrophysics
- On tit for tat: Franceschini and Maisano versus ANVUR regarding the Italian research assessment exercise VQR 2011-2014
- Qualitative Judgement of Research Impact: Domain Taxonomy as a Fundamental Framework for Judgement of the Quality of Research
- Errors and secret data in the Italian research assessment exercise. A comment to a reply
- Uncertain research country rankings. Should we continue producing uncertain rankings?
- What is the effect of country-specific characteristics on the research performance of scientific institutions? Using multi-level statistical models to rank and map universities and research-focused institutions worldwide