24 citations · 69 across the 20 of their papers we have counts for
6 papers · 1 filter
PoE: a Panel of Experts for Generalized Automatic Dialogue Assessment
Chen Zhang, Luis Fernando D'Haro, Qiquan Zhang +2
Chatbots are expected to be knowledgeable across multiple domains, e.g. for daily chit-chat, exchange of information, and grounding in emotional situations. To effectively measure…
FineD-Eval: Fine-grained Automatic Dialogue-Level Evaluation
Chen Zhang, Luis Fernando D'Haro, Qiquan Zhang +2
Recent model-based reference-free metrics for open-domain dialogue evaluation exhibit promising correlations with human judgment. However, they either perform turn-level evaluation…
A Focused Study on Sequence Length for Dialogue Summarization
Bin Wang, Chen Zhang, Chengwei Wei +1
Output length is critical to dialogue summarization systems. The dialogue summary length is determined by multiple factors, including dialogue complexity, summary objective, and pe…
Analyzing and Evaluating Faithfulness in Dialogue Summarization
Bin Wang, Chen Zhang, Yan Zhang +2
Dialogue summarization is abstractive in nature, making it suffer from factual errors. The factual correctness of summaries has the highest priority before practical applications.…
Report from the NSF Future Directions Workshop on Automatic Evaluation of Dialog: Research Directions and Challenges
Shikib Mehri, Jinho Choi, Luis Fernando D'Haro +13
This is a report on the NSF Future Directions Workshop on Automatic Evaluation of Dialog. The workshop explored the current state of the art along with its limitations and suggeste…
MDD-Eval: Self-Training on Augmented Data for Multi-Domain Dialogue Evaluation
Chen Zhang, Luis Fernando D'Haro, Thomas Friedrichs +1
Chatbots are designed to carry out human-like conversations across different domains, such as general chit-chat, knowledge exchange, and persona-grounded conversations. To measure…