344 citations · 1.7k across the 55 of their papers we have counts for
27 papers · 1 filter
Sabiá-3 Technical Report
Hugo Abonizio, Thales Sales Almeida, Thiago Laitz +4
This report presents Sabiá-3, our new flagship language model, and Sabiazinho-3, a more cost-effective sibling. The models were trained on a large brazilian-centric corpus. Evaluat…
Sabiá-2: A New Generation of Portuguese Large Language Models
Thales Sales Almeida, Hugo Abonizio, Rodrigo Nogueira +1
We introduce Sabiá-2, a family of large language models trained on Portuguese texts. The models are evaluated on a diverse range of exams, including entry-level tests for Brazilian…
BLUEX: A benchmark based on Brazilian Leading Universities Entrance eXams
Thales Sales Almeida, Thiago Laitz, Giovana K. Bonás +1
One common trend in recent studies of language models (LMs) is the use of standardized tests for evaluation. However, despite being the fifth most spoken language worldwide, few su…
Sabiá: Portuguese Large Language Models
Ramon Pires, Hugo Abonizio, Thales Sales Almeida +1
As the capabilities of language models continue to advance, it is conceivable that "one-size-fits-all" model will remain as the main paradigm. For instance, given the vast number o…
Evaluating GPT-3.5 and GPT-4 Models on Brazilian University Admission Exams
Desnes Nunes, Ricardo Primi, Ramon Pires +2
The present study aims to explore the capabilities of Language Models (LMs) in tackling high-stakes multiple-choice tests, represented here by the Exame Nacional do Ensino Médio (E…
ExaRanker: Explanation-Augmented Neural Ranker
Fernando Ferraretto, Thiago Laitz, Roberto Lotufo +1
Recent work has shown that inducing a large language model (LLM) to generate explanations prior to outputting an answer is an effective strategy to improve performance on a wide ra…
Visconde: Multi-document QA with GPT-3 and Neural Reranking
Jayr Pereira, Robson Fidalgo, Roberto Lotufo +1
This paper proposes a question-answering system that can answer questions whose supporting evidence is spread over multiple (potentially long) documents. The system, called Viscond…
mRobust04: A Multilingual Version of the TREC Robust 2004 Benchmark
Vitor Jeronymo, Mauricio Nascimento, Roberto Lotufo +1
Robust 2004 is an information retrieval benchmark whose large number of judgments per query make it a reliable evaluation dataset. In this paper, we present mRobust04, a multilingu…
MonoByte: A Pool of Monolingual Byte-level Language Models
Hugo Abonizio, Leandro Rodrigues de Souza, Roberto Lotufo +1
The zero-shot cross-lingual ability of models pretrained on multilingual and even monolingual corpora has spurred many hypotheses to explain this intriguing empirical result. Howev…
Induced Natural Language Rationales and Interleaved Markup Tokens Enable Extrapolation in Large Language Models
Mirelle Bueno, Carlos Gemmell, Jeffrey Dalton +2
The ability to extrapolate, i.e., to make predictions on sequences that are longer than those presented as training examples, is a challenging problem for current deep learning mod…
Billions of Parameters Are Worth More Than In-domain Training Data: A case study in the Legal Case Entailment Task
Guilherme Moraes Rosa, Luiz Bonifacio, Vitor Jeronymo +3
Recent work has shown that language models scaled to billions of parameters, such as GPT-3, perform remarkably well in zero-shot and few-shot scenarios. In this work, we experiment…
InPars: Data Augmentation for Information Retrieval using Large Language Models
Luiz Bonifacio, Hugo Abonizio, Marzieh Fadaee +1
The information retrieval community has recently witnessed a revolution due to large pretrained transformer models. Another key ingredient for this revolution was the MS MARCO data…
To Tune or Not To Tune? Zero-shot Models for Legal Case Entailment
Guilherme Moraes Rosa, Ruan Chaves Rodrigues, Roberto de Alencar Lotufo +1
There has been mounting evidence that pretrained language models fine-tuned on large and diverse supervised datasets can transfer well to a variety of out-of-domain tasks. In this…
On the ability of monolingual models to learn language-agnostic representations
Leandro Rodrigues de Souza, Rodrigo Nogueira, Roberto Lotufo
Pretrained multilingual models have become a de facto default approach for zero-shot cross-lingual transfer. Previous work has shown that these models are able to achieve cross-lin…
mMARCO: A Multilingual Version of the MS MARCO Passage Ranking Dataset
Luiz Bonifacio, Vitor Jeronymo, Hugo Queiroz Abonizio +4
The MS MARCO ranking dataset has been widely used for training deep learning models for IR tasks, achieving considerable effectiveness on diverse zero-shot scenarios. However, this…
A cost-benefit analysis of cross-lingual transfer methods
Guilherme Moraes Rosa, Luiz Henrique Bonifacio, Leandro Rodrigues de Souza +2
An effective method for cross-lingual transfer is to fine-tune a bilingual or multilingual model on a supervised dataset in one language and evaluating it on another language in a…
Investigating the Limitations of Transformers with Simple Arithmetic Tasks
Rodrigo Nogueira, Zhiying Jiang, Jimmy Lin
The ability to perform arithmetic tasks is a remarkable trait of human intelligence and might form a critical component of more complex reasoning tasks. In this work, we investigat…
Scientific Claim Verification with VERT5ERINI
Ronak Pradeep, Xueguang Ma, Rodrigo Nogueira +1
This work describes the adaptation of a pretrained sequence-to-sequence model to the task of scientific claim verification in the biomedical domain. We propose VERT5ERINI that expl…
Lite Training Strategies for Portuguese-English and English-Portuguese Translation
Alexandre Lopes, Rodrigo Nogueira, Roberto Lotufo +1
Despite the widespread adoption of deep learning for machine translation, it is still expensive to develop high-quality translation models. In this work, we investigate the use of…
PTT5: Pretraining and validating the T5 model on Brazilian Portuguese data
Diedre Carmo, Marcos Piau, Israel Campiotti +2
In natural language processing (NLP), there is a need for more resources in Portuguese, since much of the data used in the state-of-the-art research is in other languages. In this…
Multi-Stage Conversational Passage Retrieval: An Approach to Fusing Term Importance Estimation and Neural Query Rewriting
Sheng-Chieh Lin, Jheng-Hong Yang, Rodrigo Nogueira +3
Conversational search plays a vital role in conversational information seeking. As queries in information seeking dialogues are ambiguous for traditional ad-hoc information retriev…
Rapidly Bootstrapping a Question Answering Dataset for COVID-19
Raphael Tang, Rodrigo Nogueira, Edwin Zhang +4
We present CovidQA, the beginnings of a question answering dataset specifically designed for COVID-19, built by hand from knowledge gathered from Kaggle's COVID-19 Open Research Da…
Rapidly Deploying a Neural Search Engine for the COVID-19 Open Research Dataset: Preliminary Thoughts and Lessons Learned
Edwin Zhang, Nikhil Gupta, Rodrigo Nogueira +2
We present the Neural Covidex, a search engine that exploits the latest neural ranking architectures to provide information access to the COVID-19 Open Research Dataset curated by…
Conversational Question Reformulation via Sequence-to-Sequence Architectures and Pretrained Language Models
Sheng-Chieh Lin, Jheng-Hong Yang, Rodrigo Nogueira +3
This paper presents an empirical study of conversational question reformulation (CQR) with sequence-to-sequence architectures and pretrained language models (PLMs). We leverage PLM…