Macro F1 and Macro F1
arXiv:1911.03347
Abstract
The 'macro F1' metric is frequently used to evaluate binary, multi-class and multi-label classification problems. Yet, we find that there exist two different formulas to calculate this quantity. In this note, we show that only under rare circumstances the two computations can be considered equivalent. More specifically, one formula well 'rewards' classifiers which produce a skewed error type distribution. In fact, the difference in outcome of the two computations can be as high as 0.5. The two computations may not only diverge in their scalar result but can also lead to different classifier rankings.
arXiv comment6 pages (+ appendix), 6 figures, fixed typo
Cited by in corpus (7)
- Under-bagging Nearest Neighbors for Imbalanced Classification
- Do We Need Online NLU Tools?
- Improving type information inferred by decompilers with supervised machine learning
- Custom Deep Neural Network for 3D Covid Chest CT-scan Classification
- Sensei: Self-Supervised Sensor Name Segmentation
- Vision-Based Autonomous Drone Control using Supervised Learning in Simulation
- Communication-Computation Efficient Device-Edge Co-Inference via AutoML