The Real-World-Weight Cross-Entropy Loss Function: Modeling the Costs of Mislabeling

Abstract

In this paper, we propose a new metric to measure goodness-of-fit for classifiers, the Real World Cost function. This metric factors in information about a real world problem, such as financial impact, that other measures like accuracy or F1 do not. This metric is also more directly interpretable for users. To optimize for this metric, we introduce the Real-World- Weight Crossentropy loss function, in both binary and single-label classification variants. Both variants allow direct input of real world costs as weights. For single-label, multicategory classification, our loss function also allows direct penalization of probabilistic false positives, weighted by label, during the training of a machine learning model. We compare the design of our loss function to the binary crossentropy and categorical crossentropy functions, as well as their weighted variants, to discuss the potential for improvement in handling a variety of known shortcomings of machine learning, ranging from imbalanced classes to medical diagnostic error to reinforcement of social bias. We create scenarios that emulate those issues using the MNIST data set and demonstrate empirical results of our new loss function. Finally, we sketch a proof of this function based on Maximum Likelihood Estimation and discuss future directions.

arXiv commentSubmitted to IEEE Access

References in corpus (4)

• SMOTE: Synthetic Minority Over-sampling Technique
• Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
• Focal Loss for Dense Object Detection
• Scalable Learning of Non-Decomposable Objectives

Cited by in corpus (26)

• A survey of loss functions for semantic segmentation
• Metrics for Benchmarking and Uncertainty Quantification: Quality, Applicability, and a Path to Best Practices for Machine Learning in Chemistry
• Toward Deep Learning Based Access Control
• Generative Modeling of Turbulence
• Quantum Neuronal Sensing of Quantum Many-Body States on a 61-Qubit Programmable Superconducting Processor
• DR-Unet104 for Multimodal MRI brain tumor segmentation
• Phase Recognition in Contrast-Enhanced CT Scans based on Deep Learning and Random Sampling
• Enhancing Cross Entropy with a Linearly Adaptive Loss Function for Optimized Classification Performance
• DXM-TransFuse U-net: Dual Cross-Modal Transformer Fusion U-net for Automated Nerve Identification
• Fire-Image-DenseNet (FIDN) for predicting wildfire burnt area using remote sensing data
• SemSegLoss: A python package of loss functions for semantic segmentation
• Adaptive Multi-scale Online Likelihood Network for AI-assisted Interactive Segmentation
• A novel Deep Learning approach for one-step Conformal Prediction approximation
• Automatic defect segmentation by unsupervised anomaly learning
• Global Adaptive Filtering Layer for Computer Vision
• BLESER: Bug Localization Based on Enhanced Semantic Retrieval
• Deep Learning-based Anomaly Detection on X-ray Images of Fuel Cell Electrodes
• CAMRI Loss: Improving Recall of a Specific Class without Sacrificing Accuracy
• CardioCaps: Attention-based Capsule Network for Class-Imbalanced Echocardiogram Classification
• Towards a Multi-purpose Robotic Nursing Assistant
• Event Detection and Classification for Long Range Sensing of Elephants Using Seismic Signal
• Spatiotemporal Ground Reaction Force Analysis using Convolutional Neural Networks to Analyze Parkinsonian Gait
• IIT_kgp at FinCausal 2020, Shared Task 1: Causality Detection using Sentence Embeddings in Financial Reports
• Estimating Free Parameters in Stochastic Oscillatory Models Using a Weighted Cost Function
• Artificial intelligence prediction of stock prices using social media
• Reformulating van Rijsbergen's $F_{\beta }$ metric for weighted binary cross-entropy