To Tune or Not To Tune? Zero-shot Models for Legal Case Entailment
arXiv:2202.03120 · doi:10.1145/3462757.3466103
Abstract
There has been mounting evidence that pretrained language models fine-tuned on large and diverse supervised datasets can transfer well to a variety of out-of-domain tasks. In this work, we investigate this transfer ability to the legal domain. For that, we participated in the legal case entailment task of COLIEE 2021, in which we use such models with no adaptations to the target domain. Our submissions achieved the highest scores, surpassing the second-best team by more than six percentage points. Our experiments confirm a counter-intuitive result in the new paradigm of pretrained language models: given limited labeled data, models with little or no adaptation to the target task can be more robust to changes in the data distribution than models fine-tuned on it. Code is available at https://github.com/neuralmind-ai/coliee.
References in corpus (9)
- Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
- UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training
- Revisiting Few-sample BERT Fine-tuning
- PROP: Pre-training with Representative Words Prediction for Ad-hoc Retrieval
- Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approach
- BERT Goes to Law School: Quantifying the Competitive Advantage of Access to Large Legal Corpora in Contract Understanding
- Pyserini: An Easy-to-Use Python Toolkit to Support Replicable IR Research with Sparse and Dense Representations
- A Benchmark for Lease Contract Review
- JNLP Team: Deep Learning for Legal Processing in COLIEE 2020