4 papers
V-DyKnow: A Dynamic Benchmark for Time-Sensitive Knowledge in Vision Language Models
Seyed Mahed Mousavi, Christian Moiola, Massimo Rizzoli +2
Vision-Language Models (VLMs) are trained on data snapshots of documents, including images and texts. Their training data and evaluation benchmarks are typically static, implicitly…
[De|Re]constructing VLMs' Reasoning in Counting
Simone Alghisi, Gabriel Roccabruna, Massimo Rizzoli +2
Vision-Language Models (VLMs) have recently gained attention due to their competitive performance on multiple downstream tasks, achieved by following user-input instructions. Howev…
CIVET: Systematic Evaluation of Understanding in VLMs
Massimo Rizzoli, Simone Alghisi, Olha Khomyn +3
While Vision-Language Models (VLMs) have achieved competitive performance in various tasks, their comprehension of the underlying structure and semantics of a scene remains underst…
Will LLMs Replace the Encoder-Only Models in Temporal Relation Classification?
Gabriel Roccabruna, Massimo Rizzoli, Giuseppe Riccardi
The automatic detection of temporal relations among events has been mainly investigated with encoder-only models such as RoBERTa. Large Language Models (LLM) have recently shown pr…