8 papers
Likelihood Ranking doesn't Scale Like Prompting in LLMs
Alessandro Bondielli, Lucia Passaro, Davide Bacciu +1
LLM evaluation is commonly performed either by prompting models to produce answers or by scoring candidate outputs with likelihood-based metrics. In multiple-choice QA, however, st…
Parts-of-Speech as Emergent Categories in SAE Latent Space
Alessandro Bondielli, Lucia Passaro, Serena Auriemma +1
Sparse AutoEncoders (SAEs) offer a promising way to inspect language model representations, but it is still unclear what kind of linguistic structure their latents expose. We use p…
From Vision to Language: Investigating Causal Information Flow in Multimodal Decision-Making
Davide Testa, Hugh Mee Wong, Alessandro Lenci +2
Vision-Language Models are commonly evaluated through their final predictions, but understanding whether these decisions are grounded in visual evidence requires tracing how visual…
CLASS-IT: Conversational and Lecture-Aligned Small-Scale Instruction Tuning for BabyLMs
Luca Capone, Alessandro Bondielli, Alessandro Lenci
This work investigates whether small-scale LMs can benefit from instruction tuning. We compare conversational and question-answering instruction tuning datasets, applied either in…
BAMBI: Developing Baby Language Models for Italian
Alice Suozzi, Luca Capone, Gianluca E. Lebani +1
This paper presents BAMBI (BAby language Models Boostrapped for Italian), a series of Baby Language Models (BabyLMs) trained on data that mimic the linguistic input received by a f…
All-in-one: Understanding and Generation in Multimodal Reasoning with the MAIA Benchmark
Davide Testa, Giovanni Bonetta, Raffaella Bernardi +5
We introduce MAIA (Multimodal AI Assessment), a native-Italian benchmark designed for fine-grained investigation of the reasoning abilities of visual language models on videos. MAI…
ExpliCa: Evaluating Explicit Causal Reasoning in Large Language Models
Martina Miliani, Serena Auriemma, Alessandro Bondielli +4
Large Language Models (LLMs) are increasingly used in tasks requiring interpretive and inferential accuracy. In this paper, we introduce ExpliCa, a new dataset for evaluating LLMs…
Prompting Encoder Models for Zero-Shot Classification: A Cross-Domain Study in Italian
Serena Auriemma, Martina Miliani, Mauro Madeddu +3
Addressing the challenge of limited annotated data in specialized fields and low-resource languages is crucial for the effective use of Language Models (LMs). While most Large Lang…