What is the appropriate length of the publication period over which to assess research performance?

Abstract

National research assessment exercises are conducted in different nations over varying periods. The choice of the publication period to be observed has to address often contrasting needs: it has to ensure the reliability of the results issuing from the evaluation, but also reach the achievement of frequent assessments. In this work we attempt to identify which is the most appropriate or optimal publication period to be observed. For this, we analyze the variation of individual researchers' productivity rankings with the length of the publication period within the period 2003-2008, by the over 30,000 Italian university scientists in the hard sciences. First we analyze the variation in rankings referring to pairs of contiguous and overlapping publication periods, and show that the variations reduce markedly with periods above three years. Then we will show the strong randomness of performance rankings over publication periods under three years. We conclude that the choice of a three year publication period would seem reliable, particularly for physics, chemistry, biology and medicine.

References in corpus (5)

• A heuristic approach to author name disambiguation in bibliometrics databases for large-scale research assessments
• Assessing the varying level of impact measurement accuracy as a function of the citation window length
• National-scale research performance assessment at the individual level
• A sensitivity analysis of researchers' productivity rankings to the time of citation observation
• Measuring science: irresistible temptations, easy shortcuts and dangerous consequences

Cited by in corpus (22)

• How do you define and measure research productivity?
• The relationship among research productivity, research collaboration, and their determinants
• The combined effects of age and seniority on research performance of full professors
• Should the research performance of scientists be distinguished by gender?
• Gender bias in academic recruitment
• The effect of multidisciplinary collaborations on research diversification
• Predicting the future success of scientific publications through social network and semantic analysis
• How long do top scientists maintain their stardom? An analysis by region, gender and discipline: evidence from Italy
• Ranking research institutions by the number of highly-cited articles per scientist
• Assessing national strengths and weaknesses in research fields
• A nation's foreign and domestic professors: which have better research performance? (The Italian case)
• Diversification versus specialization in scientific research: which strategy pays off?
• Selection committees for academic recruitment: does gender matter?
• The determinants of academic career advancement: evidence from Italy
• A novel methodology to assess the scientific standing of nations at field level
• Who benefits from a country's scientific research?
• A methodology to measure the effectiveness of academic recruitment and turnover
• Comparison of research productivity of Italian and Norwegian professors and universities
• From rankings to funnel plots: the question of accounting for uncertainty when measuring university research performance
• A comparison of university performance scores and ranks by MNCS and FSS
• Funnel plots for visualizing uncertainty in the research performance of institutions
• Authorship analysis of specialized vs diversified research output