4 papers
Can't Find Waldo: Evaluating VLMs' Sensitivity to Image Resolution and Detail Level
Alexandra Schild, Gerard de Melo
Visual Language Models (VLMs) have achieved remarkable success across diverse tasks, yet they struggle with high-resolution inputs where critical information resides in small regio…
PyraMathBench: Evaluating and Improving Mathematical Capability in Large Language Models
Zetian Ouyang, Linlin Wang, Gerard de Melo +1
Despite the pivotal role of numerical reasoning as the cornerstone of mathematical capabilities in large language models (LLMs) across applications, few benchmarks evaluate LLMs by…
AutoMedEval: Harnessing Language Models for Automatic Medical Capability Evaluation
Xiechi Zhang, Zetian Ouyang, Linlin Wang +6
With the proliferation of large language models (LLMs) in the medical domain, there is increasing demand for improved evaluation techniques to assess their capabilities. However, t…
Investigating Wit, Creativity, and Detectability of Large Language Models in Domain-Specific Writing Style Adaptation of Reddit's Showerthoughts
Tolga Buz, Benjamin Frost, Nikola Genchev +3
Recent Large Language Models (LLMs) have shown the ability to generate content that is difficult or impossible to distinguish from human writing. We investigate the ability of diff…