Benchmarking Long-tail Generalization with Likelihood Splits

Ameya Godbole, Robin Jia

In order to reliably process natural language, NLP systems must generalize to the long tail of rare utterances. We propose a method to create challenging benchmarks that require ge…