The Earth Mover's Pinball Loss: Quantiles for Histogram-Valued Regression

Abstract

Although ubiquitous in the sciences, histogram data have not received much attention by the Deep Learning community. Whilst regression and classification tasks for scalar and vector data are routinely solved by neural networks, a principled approach for estimating histogram labels as a function of an input vector or image is lacking in the literature. We present a dedicated method for Deep Learning-based histogram regression, which incorporates cross-bin information and yields distributions over possible histograms, expressed by $\tau$-quantiles of the cumulative histogram in each bin. The crux of our approach is a new loss function obtained by applying the pinball loss to the cumulative histogram, which for 1D histograms reduces to the Earth Mover's distance (EMD) in the special case of the median ($\tau =0.5$), and generalizes it to arbitrary quantiles. We validate our method with an illustrative toy example, a football-related task, and an astrophysical computer vision problem. We show that with our loss function, the accuracy of the predicted median histograms is very similar to the standard EMD case (and higher than for per-bin loss functions such as cross-entropy), while the predictions become much more informative at almost no additional computational cost.

arXiv commentICML 2021. The code is available at https://github.com/FloList/EMPL

References in corpus (14)

• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• Improving neural networks by preventing co-adaptation of feature detectors
• HEALPix -- a Framework for High Resolution Discretization, and Fast Analysis of Data Distributed on the Sphere
• Fermi Large Area Telescope First Source Catalog
• On The Origin Of The Gamma Rays From The Galactic Center
• Learning Deep Embeddings with Histogram Loss
• Possible Evidence For Dark Matter Annihilation In The Inner Milky Way From The Fermi Gamma Ray Space Telescope
• Squared Earth Mover's Distance-based Loss for Training Deep Neural Networks
• Single-Model Uncertainties for Deep Learning
• Histogram Layers for Texture Analysis
• The GCE in a New Light: Disentangling the $\gamma$-ray Sky with Bayesian Graph Convolutional Neural Networks
• Analyzing γ-rays of the Galactic Center with Deep Learning
• A Novel Smoothed Loss and Penalty Function for Noncrossing Composite Quantile Estimation via Deep Neural Networks
• Continuous Histogram Loss: Beyond Neural Similarity