Performance Metrics (Error Measures) in Machine Learning Regression, Forecasting and Prognostics: Properties and Typology

Abstract

Performance metrics (error measures) are vital components of the evaluation frameworks in various fields. The intention of this study was to overview of a variety of performance metrics and approaches to their classification. The main goal of the study was to develop a typology that will help to improve our knowledge and understanding of metrics and facilitate their selection in machine learning regression, forecasting and prognostics. Based on the analysis of the structure of numerous performance metrics, we propose a framework of metrics which includes four (4) categories: primary metrics, extended metrics, composite metrics, and hybrid sets of metrics. The paper identified three (3) key components (dimensions) that determine the structure and properties of primary metrics: method of determining point distance, method of normalization, method of aggregation of point distances over a data set.

References in corpus (2)

• A better measure of relative prediction accuracy for model selection and model estimation
• Evaluating Hospital Case Cost Prediction Models Using Azure Machine Learning Studio

Cited by in corpus (9)

• Insights into Performance Fitness and Error Metrics for Machine Learning
• A Deep Learning Framework for Optimization of MISO Downlink Beamforming
• Accountability in an Algorithmic Society: Relationality, Responsibility, and Robustness in Machine Learning
• Performance of regression models as a function of experiment noise
• Explainable Software Defect Prediction from Cross Company Project Metrics Using Machine Learning
• Midwifery Learning and Forecasting: Predicting Content Demand with User-Generated Logs
• Learning-based Prediction and Uplink Retransmission for Wireless Virtual Reality (VR) Network
• Dialogue Disentanglement in Software Engineering: How Far are We?
• A Recommendation System to Enhance Midwives' Capacities in Low-Income Countries