17 citations · 45 across the 12 of their papers we have counts for
13 papers
A comparison of translation performance between DeepL and Supertext
Alex Flückiger, Chantal Amrhein, Tim Graf +5
As strong machine translation (MT) systems are increasingly based on large language models (LLMs), reliable quality benchmarking requires methods that capture their ability to leve…
Machine Translation Meta Evaluation through Translation Accuracy Challenge Sets
Nikita Moghe, Arnisa Fazla, Chantal Amrhein +5
Recent machine translation (MT) metrics calibrate their effectiveness by correlating with human judgement but without any insights about their behaviour across different error type…
A Benchmark for Evaluating Machine Translation Metrics on Dialects Without Standard Orthography
Noëmi Aepli, Chantal Amrhein, Florian Schottmann +1
For sensible progress in natural language processing, it is important that we are aware of the limitations of the evaluation metrics we use. In this work, we evaluate how robust me…
ACES: Translation Accuracy Challenge Sets at WMT 2023
Chantal Amrhein, Nikita Moghe, Liane Guillou
We benchmark the performance of segmentlevel metrics submitted to WMT 2023 using the ACES Challenge Set (Amrhein et al., 2022). The challenge set consists of 36K examples represent…
Evaluating the Effectiveness of Natural Language Inference for Hate Speech Detection in Languages with Limited Labeled Data
Janis Goldzycher, Moritz Preisig, Chantal Amrhein +1
Most research on hate speech detection has focused on English where a sizeable amount of labeled training data is available. However, to expand hate speech detection into more lang…
Exploiting Biased Models to De-bias Text: A Gender-Fair Rewriting Model
Chantal Amrhein, Florian Schottmann, Rico Sennrich +1
Natural language generation models reproduce and often amplify the biases present in their training data. Previous research explored using sequence-to-sequence rewriting models to…
Don't Discard Fixed-Window Audio Segmentation in Speech-to-Text Translation
Chantal Amrhein, Barry Haddow
For real-life applications, it is crucial that end-to-end spoken language translation models perform well on continuous audio, without relying on human-supplied segmentation. For o…
ACES: Translation Accuracy Challenge Sets for Evaluating Machine Translation Metrics
Chantal Amrhein, Nikita Moghe, Liane Guillou
As machine translation (MT) metrics improve their correlation with human judgement every year, it is crucial to understand the limitations of such metrics at the segment level. Spe…
Identifying Weaknesses in Machine Translation Metrics Through Minimum Bayes Risk Decoding: A Case Study for COMET
Chantal Amrhein, Rico Sennrich
Neural metrics have achieved impressive correlation with human judgements in the evaluation of machine translation systems, but before we can safely optimise towards such metrics,…
How Suitable Are Subword Segmentation Strategies for Translating Non-Concatenative Morphology?
Chantal Amrhein, Rico Sennrich
Data-driven subword segmentation has become the default strategy for open-vocabulary machine translation and other NLP tasks, but may not be sufficiently generic for optimal learni…
On Biasing Transformer Attention Towards Monotonicity
Annette Rios, Chantal Amrhein, Noëmi Aepli +1
Many sequence-to-sequence tasks in natural language processing are roughly monotonic in the alignment between source and target sequence, and previous work has facilitated or enfor…
On Romanization for Model Transfer Between Scripts in Neural Machine Translation
Chantal Amrhein, Rico Sennrich
Transfer learning is a popular strategy to improve the quality of low-resource machine translation. For an optimal transfer of the embedding layer, the child and parent model shoul…
Post-editing Productivity with Neural Machine Translation: An Empirical Assessment of Speed and Quality in the Banking and Finance Domain
Samuel Läubli, Chantal Amrhein, Patrick Düggelin +3
Neural machine translation (NMT) has set new quality standards in automatic translation, yet its effect on post-editing productivity is still pending thorough investigation. We emp…