Semi-analytical approximations to statistical moments of sigmoid and softmax mappings of normal variables

Abstract

This note is concerned with accurate and computationally efficient approximations of moments of Gaussian random variables passed through sigmoid or softmax mappings. These approximations are semi-analytical (i.e. they involve the numerical adjustment of parametric forms) and highly accurate (they yield 5% error at most). We also highlight a few niche applications of these approximations, which arise in the context of, e.g., drift-diffusion models of decision making or non-parametric data clustering approaches. We provide these as examples of efficient alternatives to more tedious derivations that would be needed if one was to approach the underlying mathematical issues in a more formal way. We hope that this technical note will be helpful to modellers facing similar mathematical issues, although maybe stemming from different academic prospects.

Cited by in corpus (5)

• Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awareness
• LogME: Practical Assessment of Pre-trained Models for Transfer Learning
• Ranking and Tuning Pre-trained Models: A New Paradigm for Exploiting Model Hubs
• Mean-Field Approximation to Gaussian-Softmax Integral with Application to Uncertainty Estimation
• Uncertainty Propagation in Convolutional Neural Networks: Technical Report