Algorithmic labeling in hierarchical classifications of publications: Evaluation of bibliographic fields and term weighting approaches

Abstract

Algorithmic classifications of research publications can be used to study many different aspects of the science system, such as the organization of science into fields, the growth of fields, interdisciplinarity, and emerging topics. How to label the classes in these classifications is a problem that has not been thoroughly addressed in the literature. In this study we evaluate different approaches to label the classes in algorithmically constructed classifications of research publications. We focus on two important choices: the choice of (1) different bibliographic fields and (2) different approaches to weight the relevance of terms. To evaluate the different choices, we created two baselines: one based on the Medical Subject Headings in MEDLINE and another based on the Science-Metrix journal classification. We tested to what extent different approaches yield the desired labels for the classes in the two baselines. Based on our results we recommend extracting terms from titles and keywords to label classes at high levels of granularity (e.g. topics). At low levels of granularity (e.g. disciplines) we recommend extracting terms from journal names and author addresses. We recommend the use of a new approach, term frequency to specificity ratio, to calculate the relevance of terms.

References in corpus (7)

• Efficient Estimation of Word Representations in Vector Space
• A correlated topic model of Science
• Which type of citation analysis generates the most accurate taxonomy of scientific and technical knowledge?
• Clustering scientific publications based on citation relations: A systematic comparison of different methods
• Granularity of algorithmically constructed publication-level classifications of research publications: Identification of topics
• Granularity of algorithmically constructed publication-level classifications of research publications: Identification of specialties
• Contextualization of topics: Browsing through the universe of bibliographic information

Cited by in corpus (4)

• The association between topic growth and citation impact of research publications
• Empirical analysis of recent temporal dynamics of research fields: Annual publications in chemistry and related areas as an example
• Improving overlay maps of science: combining overview and detail
• Using language models to label clusters of scientific documents