5 papers
When Fairness Isn't Statistical: The Limits of Machine Learning in Evaluating Legal Reasoning
Claire Barale, Michael Rovatsos, Nehal Bhuta
Legal decisions are increasingly evaluated for fairness, consistency, and bias using machine learning (ML) techniques. In high-stakes domains like refugee adjudication, such method…
LexTime: A Benchmark for Temporal Ordering of Legal Events
Claire Barale, Leslie Barrett, Vikram Sunil Bajaj +1
Understanding temporal relationships and accurately reconstructing the event timeline is important for case law analysis, compliance monitoring, and legal summarization. However, e…
Are We Done with MMLU?
Aryo Pradipta Gema, Joshua Ong Jun Leang, Giwon Hong +13
Maybe not. We identify and analyse errors in the popular Massive Multitask Language Understanding (MMLU) benchmark. Even though MMLU is widely adopted, our analysis demonstrates nu…
Do Language Models Learn about Legal Entity Types during Pretraining?
Claire Barale, Michael Rovatsos, Nehal Bhuta
Language Models (LMs) have proven their ability to acquire diverse linguistic knowledge during the pretraining phase, potentially serving as a valuable source of incidental supervi…
Empowering Refugee Claimants and their Lawyers: Using Machine Learning to Examine Decision-Making in Refugee Law
Claire Barale
Our project aims at helping and supporting stakeholders in refugee status adjudications, such as lawyers, judges, governing bodies, and claimants, in order to make better decisions…