The Validity of Evaluation Results: Assessing Concurrence Across Compositionality Benchmarks

Kaiser Sun, Adina Williams, Dieuwke Hupkes

NLP models have progressed drastically in recent years, according to numerous datasets proposed to evaluate performance. Questions remain, however, about how particular dataset des…