On The Statistical Representation Properties Of The Perturb-Softmax And The Perturb-Argmax Probability Distributions

Hedda Cohen Indelman, Tamir Hazan

The Gumbel-Softmax probability distribution allows learning discrete tokens in generative learning, while the Gumbel-Argmax probability distribution is useful in learning discrete…