20 citations · 24 across the 7 of their papers we have counts for
3 papers · 1 filter
How much pretraining data do language models need to learn syntax?
Laura Pérez-Mayos, Miguel Ballesteros, Leo Wanner
Transformers-based pretrained language models achieve outstanding results in many well-known NLU benchmarks. However, while pretraining methods are very convenient, they are expens…
Assessing the Syntactic Capabilities of Transformer-based Multilingual Language Models
Laura Pérez-Mayos, Alba Táboas García, Simon Mille +1
Multilingual Transformer-based language models, usually pretrained on more than 100 languages, have been shown to achieve outstanding results in a wide range of cross-lingual trans…
On the Evolution of Syntactic Information Encoded by BERT's Contextualized Representations
Laura Pérez-Mayos, Roberto Carlini, Miguel Ballesteros +1
The adaptation of pretrained language models to solve supervised tasks has become a baseline in NLP, and many recent works have focused on studying how linguistic information is en…