On tit for tat: Franceschini and Maisano versus ANVUR regarding the Italian research assessment exercise VQR 2011-2014
arXiv:1810.12635 · doi:10.1016/j.joi.2017.06.003
Abstract
The response by Benedetto, Checchi, Graziosi & Malgarini (2017) (hereafter "BCG&M"), past and current members of the Italian Agency for Evaluation of University and Research Systems (ANVUR), to Franceschini and Maisano's ("F&M") article (2017), inevitably draws us into the debate. BCG&M in fact complain "that almost all criticisms to the evaluation procedures adopted in the two Italian research assessments VQR 2004-2010 and 2011-2014 limit themselves to criticize the procedures without proposing anything new and more apt to the scope". Since it is us who raised most criticisms in the literature, we welcome this opportunity to retrace our vainly "constructive" recommendations, made with the hope of contributing to assessments of the Italian research system more in line with the state of the art in scientometrics. We see it as equally interesting to confront the problem of the failure of knowledge transfer from R&D (scholars) to engineering and production (ANVUR's practitioners) in the Italian VQRs. We will provide a few notes to help the reader understand the context for this failure. We hope that these, together with our more specific comments, will also assist in communicating the reasons for the level of scientometric competence expressed in BCG&M's heated response to F&M's criticism.
References in corpus (12)
- Evaluating research: from informed peer review to bibliometrics
- Allocative efficiency in public research funding:can bibliometrics help?
- Research productivity: are higher academic ranks more productive than lower ones?
- The importance of accounting for the number of co-authors and their order when assessing research performance at the individual level in the life sciences
- National research assessment exercises: a comparison of peer review and bibliometrics rankings
- The VQR, Italy's second national research assessment: Methodological failures and ranking distortions
- National peer-review research assessment exercises for the hard sciences can be a complete waste of money: the Italian case
- Refrain from adopting the combination of citation and journal metrics to grade publications, as used in the Italian national research assessment exercise (VQR 2011-2014)
- Measuring institutional research productivity for the life sciences: the importance of accounting for the order of authors in the byline
- Inefficiency in selecting products for submission to national research assessment exercises
- National research assessment exercises: a measure of the distortion of performance rankings when labor input is treated as uniform
- Peer review research assessment: a sensitivity analysis of performance rankings to the share of research product evaluated