activity
20212026
most citedEfficient pre-training objectives for Transformers

4 citations · 4 across the 8 of their papers we have counts for

collaborators

8 papers

cs.CL2026

Multilingual Training and Evaluation Resources for Vision-Language Models

Daniela Baiamonte, Elena Fano, Matteo Gabburo +3

Vision Language Models (VLMs) achieved rapid progress in the recent years. However, despite their growth, VLMs development is heavily grounded on English, leading to two main limit…

cs.CL2024

Datasets for Multilingual Answer Sentence Selection

Matteo Gabburo, Stefano Campese, Federico Agostini +1

Answer Sentence Selection (AS2) is a critical task for designing effective retrieval-based Question Answering (QA) systems. Most advancements in AS2 focus on English due to the sca…

cs.CL2024

Measuring Retrieval Complexity in Question Answering Systems

Matteo Gabburo, Nicolaas Paul Jedema, Siddhant Garg +2

In this paper, we investigate which questions are challenging for retrieval-based Question Answering (QA). We (i) propose retrieval complexity (RC), a novel metric conditioned on t…

cs.CL2023

SQUARE: Automatic Question Answering Evaluation using Multiple Positive and Negative References

Matteo Gabburo, Siddhant Garg, Rik Koncel Kedziorski +1

Evaluation of QA systems is very challenging and expensive, with the most reliable approach being human annotations of correctness of answers for questions. Recent works (AVA, BEM)…

cs.CL2023

Learning Answer Generation using Supervision from Automatic Question Answering Evaluators

Matteo Gabburo, Siddhant Garg, Rik Koncel-Kedziorski +1

Recent studies show that sentence-level extractive QA, i.e., based on Answer Sentence Selection (AS2), is outperformed by Generation-based QA (GenQA) models, which generate answers…

cs.CL2022

Effective Pre-Training Objectives for Transformer-based Autoencoders

Luca Di Liello, Matteo Gabburo, Alessandro Moschitti

In this paper, we study trade-offs between efficiency, cost and accuracy when pre-training Transformer encoders with different pre-training objectives. For this purpose, we analyze…

cs.CL2022

Knowledge Transfer from Answer Ranking to Answer Generation

Matteo Gabburo, Rik Koncel-Kedziorski, Siddhant Garg +2

Recent studies show that Question Answering (QA) based on Answer Sentence Selection (AS2) can be improved by generating an improved answer from the top-k ranked answer sentences (t…

cs.CL2021★ 4 cited

Efficient pre-training objectives for Transformers

Luca Di Liello, Matteo Gabburo, Alessandro Moschitti

The Transformer architecture deeply changed the natural language processing, outperforming all previous state-of-the-art models. However, well-known Transformer models like BERT, R…