4 citations · 4 across the 7 of their papers we have counts for
3 papers · 1 filter
Effective Pre-Training Objectives for Transformer-based Autoencoders
Luca Di Liello, Matteo Gabburo, Alessandro Moschitti
In this paper, we study trade-offs between efficiency, cost and accuracy when pre-training Transformer encoders with different pre-training objectives. For this purpose, we analyze…
Knowledge Transfer from Answer Ranking to Answer Generation
Matteo Gabburo, Rik Koncel-Kedziorski, Siddhant Garg +2
Recent studies show that Question Answering (QA) based on Answer Sentence Selection (AS2) can be improved by generating an improved answer from the top-k ranked answer sentences (t…
Efficient pre-training objectives for Transformers
Luca Di Liello, Matteo Gabburo, Alessandro Moschitti
The Transformer architecture deeply changed the natural language processing, outperforming all previous state-of-the-art models. However, well-known Transformer models like BERT, R…