3 citations · 8 across the 24 of their papers we have counts for
9 papers · 1 filter
MindEval2: Clinically Grounded Patients and Rubrics for Evaluating Language Models in Mental Health Support
John Mendonça, Daniel Tschernutter, José Pombal +6
With people increasingly turning to AI chatbots for mental health support, the need to evaluate these tools is becoming more pressing. Recent benchmarks attempt to mimic a realisti…
DEPICT: Scoring Text-to-Image Alignment by Answer Agreement
Vasco Ramos, Sandra Godinho Silva, Joao Magalhaes +2
Image-text alignment is a core problem in computer vision with applications in caption evaluation, hallucination detection, data curation, and the benchmarking of text-to-image (T2…
IndicQE-APE: A Consolidated Benchmark for Quality Estimation and Automatic Post-Editing for Indic Languages
Diptesh Kanojia, Archchana Sindhujan, Sourabh Deoghare +14
Indic quality estimation (QE) and automatic post-editing (APE) data is spread across separate releases, so no single resource supports training and evaluation across tasks and lang…
Move by Move: Measuring and Steering How LLMs Conduct Psychotherapy
Afonso Baldo, Hugo Pitorro, Areti Vassilopoulos +5
As the demand for mental health services outpaces clinical workforce capacity, users are increasingly turning to large language models (LLMs) for accessible and private support. Al…
Risk Governance for Generative AI Mental Health Support: A Multi-Turn Safety Architecture
Anabela C. Areias, Catarina Botelho, António Farinhas +7
Large language models (LLMs) are increasingly used for emotional support despite lacking mechanisms to safely govern evolving mental health risk. Existing safety approaches primari…
Can Vision Language Models Judge Action Quality? An Empirical Evaluation
Miguel Monte e Freitas, Rui Henriques, Ricardo Rei +1
Action Quality Assessment (AQA) has broad applications in physical therapy, sports coaching, and competitive judging. Although Vision Language Models (VLMs) hold considerable promi…
Self-Preference Bias in Rubric-Based Evaluation of Large Language Models
José Pombal, Ricardo Rei, André F. T. Martins
LLM-as-a-judge has become the de facto approach for evaluating LLM outputs. However, judges are known to exhibit self-preference bias (SPB): they tend to favor outputs produced by…
EuroLLM-22B: Technical Report
Miguel Moura Ramos, Duarte M. Alves, Hippolyte Gisserot-Boukhlef +15
This report presents EuroLLM-22B, a large language model trained from scratch to support the needs of European citizens by covering all 24 official European Union languages and 11…
MindGuard: Guardrail Classifiers for Multi-Turn Mental Health Support
António Farinhas, Nuno M. Guerreiro, José Pombal +6
Large language models are increasingly used for mental health support, yet their conversational coherence alone does not ensure clinical appropriateness. Existing general-purpose s…