Evaluating Automatic Metrics with Incremental Machine Translation Systems

Guojun Wu, Shay B. Cohen, Rico Sennrich

We introduce a dataset comprising commercial machine translations, gathered weekly over six years across 12 translation directions. Since human A/B testing is commonly used, we ass…