A sensitivity analysis of researchers' productivity rankings to the time of citation observation
arXiv:1811.01658 · doi:10.1016/j.joi.2011.12.003
Abstract
In this work we investigate the sensitivity of individual researchers' productivity rankings to the time of citation observation. The analysis is based on observation of research products for the 2001-2003 triennium for all research staff of Italian universities in the hard sciences, with the year of citation observation varying from 2004 to 2008. The 2008 rankings list is assumed the most accurate, as citations have had the longest time to accumulate and thus represent the best possible proxy of impact. By comparing the rankings lists from each year against the 2008 benchmark we provide policy-makers and research organization managers a measure of trade-off between timeliness of evaluation execution and accuracy of performance rankings. The results show that with variation in the evaluation citation window there are variable rates of inaccuracy across the disciplines of researchers. The inaccuracy results negligible for Physics, Biology and Medicine.
References in corpus (7)
- Evaluating research: from informed peer review to bibliometrics
- A heuristic approach to author name disambiguation in bibliometrics databases for large-scale research assessments
- Allocative efficiency in public research funding:can bibliometrics help?
- Assessing the varying level of impact measurement accuracy as a function of the citation window length
- A field-standardized application of DEA to national-scale research assessment of universities
- National-scale research performance assessment at the individual level
- A sensitivity analysis of research institutions' productivity rankings to the time of citation observation
Cited by in corpus (4)
- How do you define and measure research productivity?
- What is the appropriate length of the publication period over which to assess research performance?
- The suitability of h and g indexes for measuring the research performance of institutions. Scientometrics, 97(3), 555-570
- Ranking research institutions by the number of highly-cited articles per scientist