A comparison of university performance scores and ranks by MNCS and FSS
arXiv:1810.12661 · doi:10.1016/j.joi.2016.07.004
Abstract
In a previous article of ours, we explained the reasons why the MNCS and all similar per-publication citation indicators should not be used to measure research performance, whereas efficiency indicators (output to input) such as the FSS are valid indicators of performance. The problem frequently indicated in measuring efficiency indicators lies in the availability of input data. If we accept that such data are inaccessible, and instead resort to per-publication citation indicators, the question arises as to what extent institution performance rankings by MNCS are different from those by FSS (and so what effects such results could have on policy-makers, managers and other users of the rankings). Contrasting the 2008-2012 performance by MNCS and FSS of Italian universities in the Sciences, we try to answer that question at field, discipline, and overall university level. We present the descriptive statistics of the shifts in rank, and the correlations of both scores and ranks. The analysis reveals strong correlations in many fields but weak correlations in others. The extent of rank shifts is never negligible: a number of universities shift from top to non-top quartile ranks.
References in corpus (14)
- How do you define and measure research productivity?
- A heuristic approach to author name disambiguation in bibliometrics databases for large-scale research assessments
- The measurement of italian universities' research productivity by a non parametric-bibliometric methodology
- A farewell to the MNCS and like size-independent indicators
- Revisiting the scaling of citations for research assessment
- What is the appropriate length of the publication period over which to assess research performance?
- Revisiting size effects in higher education research productivity
- Testing the trade-off between productivity and quality in research activities
- Measuring institutional research productivity for the life sciences: the importance of accounting for the order of authors in the byline
- The suitability of h and g indexes for measuring the research performance of institutions. Scientometrics, 97(3), 555-570
- Evaluating university research: same performance indicator, different rankings
- Ranking research institutions by the number of highly-cited articles per scientist
- Assessing the accuracy of the h and g indexes for measuring researchers' productivity
- National research assessment exercises: a measure of the distortion of performance rankings when labor input is treated as uniform