On Mean Absolute Error for Deep Neural Network Based Vector-to-Vector Regression

Abstract

In this paper, we exploit the properties of mean absolute error (MAE) as a loss function for the deep neural network (DNN) based vector-to-vector regression. The goal of this work is two-fold: (i) presenting performance bounds of MAE, and (ii) demonstrating new properties of MAE that make it more appropriate than mean squared error (MSE) as a loss function for DNN based vector-to-vector regression. First, we show that a generalized upper-bound for DNN-based vector- to-vector regression can be ensured by leveraging the known Lipschitz continuity property of MAE. Next, we derive a new generalized upper bound in the presence of additive noise. Finally, in contrast to conventional MSE commonly adopted to approximate Gaussian errors for regression, we show that MAE can be interpreted as an error modeled by Laplacian distribution. Speech enhancement experiments are conducted to corroborate our proposed theorems and validate the performance advantages of MAE over MSE for DNN based regression.

References in corpus (3)

• One pixel attack for fooling deep neural networks
• Local Rademacher complexities
• Analyzing Upper Bounds on Mean Absolute Errors for Deep Neural Network Based Vector-to-Vector Regression

Cited by in corpus (6)

• An Adaptive and Robust Deep Learning Framework for THz Ultra-Massive MIMO Channel Estimation
• An Integrated Framework for the Heterogeneous Spatio-Spectral-Temporal Fusion of Remote Sensing Images
• Symbolic Regression via Neural Networks
• Learning Aerodynamics for the Control of Flying Humanoid Robots
• Data-Driven Reduction of Fault Location Errors in Onshore Wind Farm Collectors
• About the Influence of Workflow Topology on Task Intensity Prediction through Graph Learning