On Degrees of Freedom in Defining and Testing Natural Language Understanding

Saku Sugawara, Shun Tsugita

Natural language understanding (NLU) studies often exaggerate or underestimate the capabilities of systems, thereby limiting the reproducibility of their findings. These erroneous…