Funnel plots for visualizing uncertainty in the research performance of institutions

Abstract

Research performance values are not certain. Performance indexes should therefore be accompanied by uncertainty measures, to establish whether the performance of a unit is truly outstanding and not the result of random fluctuations. In this work we focus on the evaluation of research institutions on the basis of average individual performance, where uncertainty is inversely related to the number of research staff. We utilize the funnel plot, a tool originally developed in meta-analysis, to measure and visualize the uncertainty in the performance values of research institutions. As an illustrative example, we apply the funnel plot to represent the uncertainty in the assessed research performance for Italian universities active in biochemistry

arXiv commentarXiv admin note: text overlap with arXiv:1810.12664

References in corpus (11)

• Universality of citation distributions: towards an objective measure of scientific impact
• How do you define and measure research productivity?
• A heuristic approach to author name disambiguation in bibliometrics databases for large-scale research assessments
• Assessing the varying level of impact measurement accuracy as a function of the citation window length
• The importance of accounting for the number of co-authors and their order when assessing research performance at the individual level in the life sciences
• Revisiting the scaling of citations for research assessment
• What is the appropriate length of the publication period over which to assess research performance?
• Caveats for using statistical significance tests in research assessments
• Revisiting size effects in higher education research productivity
• The dispersion of research performance within and between universities as a potential indicator of the competitive intensity in higher education systems
• A national-scale cross-time analysis of university research performance

Cited by in corpus (4)

• A methodology to measure the effectiveness of academic recruitment and turnover
• Conceptual difficulties in the use of statistical inference in citation analysis
• From rankings to funnel plots: the question of accounting for uncertainty when measuring university research performance
• The Research Production of Nations and Departments: A Statistical Model for the Share of Publications