Evaluation Measures for Hierarchical Classification: a unified view and novel approaches
arXiv:1306.6802 · doi:10.1007/s10618-014-0382-x
Abstract
Hierarchical classification addresses the problem of classifying items into a hierarchy of classes. An important issue in hierarchical classification is the evaluation of different classification algorithms, which is complicated by the hierarchical relations among the classes. Several evaluation measures have been proposed for hierarchical classification using the hierarchy in different ways. This paper studies the problem of evaluation in hierarchical classification by analyzing and abstracting the key components of the existing performance measures. It also proposes two alternative generic views of hierarchical evaluation and introduces two corresponding novel measures. The proposed measures, along with the state-of-the art ones, are empirically tested on three large datasets from the domain of text classification. The empirical results illustrate the undesirable behavior of existing approaches and how the proposed methods overcome most of these methods across a range of cases.
Submitted to journal
Cited by in corpus (10)
- An Analysis of Hierarchical Text Classification Using Word Embeddings
- A Closer Look at Classification Evaluation Metrics and a Critical Reflection of Common Evaluation Practice
- Hierarchical confusion matrix for classification performance evaluation
- Overview of BioASQ 2021: The ninth BioASQ challenge on Large-Scale Biomedical Semantic Indexing and Question Answering
- Overview of BioASQ 2020: The eighth BioASQ challenge on Large-Scale Biomedical Semantic Indexing and Question Answering
- Can GPT-3.5 Generate and Code Discharge Summaries?
- Results of the seventh edition of the BioASQ Challenge
- Overview of BioASQ 2022: The tenth BioASQ challenge on Large-Scale Biomedical Semantic Indexing and Question Answering
- HunFlair2 in a cross-corpus evaluation of biomedical named entity recognition and normalization tools
- Hierarchical Scoring for Machine Learning Classifier Error Impact Evaluation