Evaluating university research: same performance indicator, different rankings
arXiv:1810.12732 · doi:10.1016/j.joi.2015.04.002
Abstract
Assessing the research performance of multi-disciplinary institutions, where scientists belong to many fields, requires that the evaluators plan how to aggregate the performance measures of the various fields. Two methods of aggregation are possible. These are based on: a) the performance of the individual scientists or b) the performance of the scientific fields present in the institution. The appropriate choice depends on the evaluation context and the objectives for the particular measure. The two methods bring about differences in both the performance scores and rankings. We quantify these differences through observation of the 2008-2012 scientific production of the entire research staff employed in the hard sciences in Italian universities (over 35,000 professors). Evaluators preparing an exercise must comprehend the differences illustrated, in order to correctly select the methodologies that will achieve the evaluation objectives.
References in corpus (9)
- How do you define and measure research productivity?
- A heuristic approach to author name disambiguation in bibliometrics databases for large-scale research assessments
- The importance of accounting for the number of co-authors and their order when assessing research performance at the individual level in the life sciences
- Revisiting the scaling of citations for research assessment
- The impact of unproductive and top researchers on overall university research performance
- The dangers of performance-based research funding in non-competitive higher education systems
- The dispersion of research performance within and between universities as a potential indicator of the competitive intensity in higher education systems
- Measuring institutional research productivity for the life sciences: the importance of accounting for the order of authors in the byline
- National research assessment exercises: a measure of the distortion of performance rankings when labor input is treated as uniform