24 citations · 24 across the 4 of their papers we have counts for
7 papers · 1 filter
A Measure of the System Dependence of Automated Metrics
Pius von Däniken, Jan Deriu, Mark Cieliebak
Automated metrics for Machine Translation have made significant progress, with the goal of replacing expensive and time-consuming human evaluations. These metrics are typically ass…
On the Effectiveness of Automated Metrics for Text Generation Systems
Pius von Däniken, Jan Deriu, Don Tuggener +1
A major challenge in the field of Text Generation is evaluation because we lack a sound theory that can be leveraged to extract guidelines for evaluation campaigns. In this work, w…
SDS-200: A Swiss German Speech to Standard German Text Corpus
Michel Plüss, Manuela Hürlimann, Marc Cuny +10
We present SDS-200, a corpus of Swiss German dialectal speech with Standard German text translations, annotated with dialect, age, and gender information of the speakers. The datas…
Report from the NSF Future Directions Workshop on Automatic Evaluation of Dialog: Research Directions and Challenges
Shikib Mehri, Jinho Choi, Luis Fernando D'Haro +13
This is a report on the NSF Future Directions Workshop on Automatic Evaluation of Dialog. The workshop explored the current state of the art along with its limitations and suggeste…
DoQA -- Accessing Domain-Specific FAQs via Conversational QA
Jon Ander Campos, Arantxa Otegi, Aitor Soroa +3
The goal of this work is to build conversational Question Answering (QA) interfaces for the large body of domain-specific information available in FAQ sites. We present DoQA, a dat…
Survey on Evaluation Methods for Dialogue Systems
Jan Deriu, Alvaro Rodrigo, Arantxa Otegi +4
In this paper we survey the methods and concepts developed for the evaluation of dialogue systems. Evaluation is a crucial part during the development process. Often, dialogue syst…