Prudence When Assuming Normality: an advice for machine learning practitioners

Abstract

In a binary classification problem the feature vector (predictor) is the input to a scoring function that produces a decision value (score), which is compared to a particular chosen threshold to provide a final class prediction (output). Although the normal assumption of the scoring function is important in many applications, sometimes it is severely violated even under the simple multinormal assumption of the feature vector. This article proves this result mathematically with a counter example to provide an advice for practitioners to avoid blind assumptions of normality. On the other hand, the article provides a set of experiments that illustrate some of the expected and well-behaved results of the Area Under the ROC curve (AUC) under the multinormal assumption of the feature vector. Therefore, the message of the article is not to avoid the normal assumption of either the input feature vector or the output scoring function; however, a prudence is needed when adopting either of both.

Cited by in corpus (5)

• Classifier Calibration: with application to threat scores in cybersecurity
• Matlab vs. OpenCV: A Comparative Study of Different Machine Learning Algorithms
• Assessment of Multiple-Biomarker Classifiers: fundamental principles and a proposed strategy
• Machine Learning Construction: implications to cybersecurity
• Nested Cavity Classifier: performance and remedy