Bringing order into the realm of Transformer-based language models for artificial intelligence and law
arXiv:2308.05502 · doi:10.1007/s10506-023-09374-7
Abstract
Transformer-based language models (TLMs) have widely been recognized to be a cutting-edge technology for the successful development of deep-learning-based solutions to problems and applications that require natural language processing and understanding. Like for other textual domains, TLMs have indeed pushed the state-of-the-art of AI approaches for many tasks of interest in the legal domain. Despite the first Transformer model being proposed about six years ago, there has been a rapid progress of this technology at an unprecedented rate, whereby BERT and related models represent a major reference, also in the legal domain. This article provides the first systematic overview of TLM-based methods for AI-driven problems and tasks in the legal sphere. A major goal is to highlight research advances in this field so as to understand, on the one hand, how the Transformers have contributed to the success of AI in supporting legal processes, and on the other hand, what are the current limitations and opportunities for further research development.
Please refer to the published version: Greco, C.M., Tagarelli, A. (2023) Bringing order into the realm of Transformer-based language models for artificial intelligence and law. Artif Intell Law, Springer Nature. November 2023. https://doi.org/10.1007/s10506-023-09374-7
References in corpus (18)
- MAGNN: Metapath Aggregated Graph Neural Network for Heterogeneous Graph Embedding
- A Deep Relevance Matching Model for Ad-hoc Retrieval
- CamemBERT: a Tasty French Language Model
- Unsupervised Learning of Sentence Embeddings using Compositional n-Gram Features
- Revisiting Pre-Trained Models for Chinese Natural Language Processing
- CEDR: Contextualized Embeddings for Document Ranking
- CLAUDETTE: an Automated Detector of Potentially Unfair Clauses in Online Terms of Service
- Pretrained Language Models for Sequential Sentence Classification
- Word-Entity Duet Representations for Document Ranking
- GREEK-BERT: The Greeks visiting Sesame Street
- Privacy Policies over Time: Curation and Analysis of a Million-Document Dataset
- Unsupervised Law Article Mining based on Deep Pre-Trained Language Representation Models with Application to the Italian Civil Code
- Explainable Legal Case Matching via Inverse Optimal Transport-based Rationale Extraction
- Structural Text Segmentation of Legal Documents
- Towards an Open Platform for Legal Information
- Lex Rosetta: Transfer of Predictive Models Across Languages, Jurisdictions, and Legal Domains
- To Tune or Not To Tune? Zero-shot Models for Legal Case Entailment
- verBERT: Automating Brazilian Case Law Document Multi-label Categorization Using BERT