verBERT: Automating Brazilian Case Law Document Multi-label Categorization Using BERT

Abstract

In this work, we carried out a study about the use of attention-based algorithms to automate the categorization of Brazilian case law documents. We used data from the Kollemata Project to produce two distinct datasets with adequate class systems. Then, we implemented a multi-class and multi-label version of BERT and fine-tuned different BERT models with the produced datasets. We evaluated several metrics, adopting the micro-averaged F1-Score as our main metric for which we obtained a performance value of F1-micro=0.72 corresponding to gains of 30 percent points over the tested statistical baseline. In this work, we carried out a study about the use of attention-based algorithms to automate the categorization of Brazilian case law documents. We used data from the \textit{Kollemata} Project to produce two distinct datasets with adequate class systems. Then, we implemented a multi-class and multi-label version of BERT and fine-tuned different BERT models with the produced datasets. We evaluated several metrics, adopting the micro-averaged F1-Score as our main metric for which we obtained a performance value of $⟨{\mathcal{F}}_1{⟩}_{micro}=0.72$ corresponding to gains of 30 percent points over the tested statistical baseline.

arXiv comment10 pages, 2 tables

References in corpus (6)

• Decoupled Weight Decay Regularization
• Machine Learning in Automated Text Categorization
• HuggingFace's Transformers: State-of-the-art Natural Language Processing
• GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
• Portuguese Named Entity Recognition using BERT-CRF
• Toxic Language Detection in Social Media for Brazilian Portuguese: New Dataset and Multilingual Analysis

Cited by in corpus (1)

• Bringing order into the realm of Transformer-based language models for artificial intelligence and law