PeLLE: Encoder-based language models for Brazilian Portuguese based on open data

Guilherme Lamartine de Mello, Marcelo Finger, and Felipe Serras +4

In this paper we present PeLLE, a family of large language models based on the RoBERTa architecture, for Brazilian Portuguese, trained on curated, open data from the Carolina corpu…