Two-Level Softmax Sampling Done Right: Correcting Bias from Size Imbalance and Dispersion

Walid Bendada, Guillaume Salha-Galvan

Sampling from a softmax distribution is a fundamental operation in machine learning, but its linear complexity in the number of items makes exact sampling impractical at scale. Two…