Towards Probabilistically-Sound Beam Search with Masked Language Models

Creston Brooks, Robert Calef, Charlie Cowen-Breen +1

Beam search with masked language models (MLMs) is challenging in part because joint probability distributions over sequences are not readily available, unlike for autoregressive mo…