1 citations · 2 across the 5 of their papers we have counts for
5 papers · 1 filter
MindEval2: Clinically Grounded Patients and Rubrics for Evaluating Language Models in Mental Health Support
John Mendonça, Daniel Tschernutter, José Pombal +6
With people increasingly turning to AI chatbots for mental health support, the need to evaluate these tools is becoming more pressing. Recent benchmarks attempt to mimic a realisti…
Soda-Eval: Open-Domain Dialogue Evaluation in the age of LLMs
John Mendonça, Isabel Trancoso, Alon Lavie
Although human evaluation remains the gold standard for open-domain dialogue evaluation, the growing popularity of automated evaluation using Large Language Models (LLMs) has also…
ECoh: Turn-level Coherence Evaluation for Multilingual Dialogues
John Mendonça, Isabel Trancoso, Alon Lavie
Despite being heralded as the new standard for dialogue evaluation, the closed-source nature of GPT-4 poses challenges for the community. Motivated by the need for lightweight, ope…
On the Benchmarking of LLMs for Open-Domain Dialogue Evaluation
John Mendonça, Alon Lavie, Isabel Trancoso
Large Language Models (LLMs) have showcased remarkable capabilities in various Natural Language Processing tasks. For automatic open-domain dialogue evaluation in particular, LLMs…
Towards Multilingual Automatic Dialogue Evaluation
John Mendonça, Alon Lavie, Isabel Trancoso
The main limiting factor in the development of robust multilingual dialogue evaluation metrics is the lack of multilingual data and the limited availability of open sourced multili…