4 papers · 1 filter
Every Wrong Answer Counts: Option-Level Psychometrics for LLM Multiple-Choice Benchmarks
Xiao Fei, Yang Zhang, Sarah Almeida Carneiro +1
Most multiple-choice question (MCQ) benchmarks evaluate Large Language Models (LLMs) only by whether they select the correct answers. This binary scoring treats all incorrect respo…
The Masked Advantage: Uncovering Local-Language Access to Cultural Knowledge in LLMs
Yang Zhang, Xiao Fei, Amr Mohamed +6
Large language models are increasingly used to answer culturally grounded questions across languages, yet it remains unclear whether local cultural knowledge is better accessed thr…
CARTE: A Benchmark for Mapping Language Model Knowledge Across France
Sarah Almeida Carneiro, Christos Xypolopoulos, Xiao Fei +2
We introduce CARTE (Culturally Anchored Regional-Territorial Evaluation), a multiple-choice benchmark for evaluating the ability of large language models (LLMs) to discriminate fin…
PPI2Text: Captioning Protein-Protein Interactions with Coordinate-Aligned Pair-Map Decoding
Xiao Fei, Sarah Almeida Carneiro, Yang Zhang +4
Protein-protein interaction (PPI) modeling has been widely studied as a binary or multi-label classification task. While emerging multimodal large language models (LLMs) can now de…