6 papers · 1 filter
Likelihood Ranking doesn't Scale Like Prompting in LLMs
Alessandro Bondielli, Lucia Passaro, Davide Bacciu +1
LLM evaluation is commonly performed either by prompting models to produce answers or by scoring candidate outputs with likelihood-based metrics. In multiple-choice QA, however, st…
Parts-of-Speech as Emergent Categories in SAE Latent Space
Alessandro Bondielli, Lucia Passaro, Serena Auriemma +1
Sparse AutoEncoders (SAEs) offer a promising way to inspect language model representations, but it is still unclear what kind of linguistic structure their latents expose. We use p…
Charting a Decade of Computational Linguistics in Italy: The CLiC-it Corpus
Chiara Alzetta, Serena Auriemma, Alessandro Bondielli +5
Over the past decade, Computational Linguistics (CL) and Natural Language Processing (NLP) have evolved rapidly, especially with the advent of Transformer-based Large Language Mode…
All-in-one: Understanding and Generation in Multimodal Reasoning with the MAIA Benchmark
Davide Testa, Giovanni Bonetta, Raffaella Bernardi +5
We introduce MAIA (Multimodal AI Assessment), a native-Italian benchmark designed for fine-grained investigation of the reasoning abilities of visual language models on videos. MAI…
ExpliCa: Evaluating Explicit Causal Reasoning in Large Language Models
Martina Miliani, Serena Auriemma, Alessandro Bondielli +4
Large Language Models (LLMs) are increasingly used in tasks requiring interpretive and inferential accuracy. In this paper, we introduce ExpliCa, a new dataset for evaluating LLMs…
Prompting Encoder Models for Zero-Shot Classification: A Cross-Domain Study in Italian
Serena Auriemma, Martina Miliani, Mauro Madeddu +3
Addressing the challenge of limited annotated data in specialized fields and low-resource languages is crucial for the effective use of Language Models (LMs). While most Large Lang…