369 citations · 1.2k across the 41 of their papers we have counts for
18 papers · 1 filter
DLawBench: Evaluating LLMs Through Multi-Turn Legal Consultation
Li Zhang, Yuzhen Shi, Yiran Hu +15
Lawyer-client consultation is a critical starting point for legal services. Effective legal assistance hinges on eliciting sufficient and truthful information from clients in order…
Do LLMs Truly Understand When a Precedent Is Overruled?
Li Zhang, Jaromir Savelka, Kevin Ashley
Large language models (LLMs) with extended context windows show promise for complex legal reasoning tasks, yet their ability to understand long legal documents remains insufficient…
Measuring Faithfulness and Abstention: An Automated Pipeline for Evaluating LLM-Generated 3-ply Case-Based Legal Arguments
Li Zhang, Morgan Gray, Jaromir Savelka +1
Large Language Models (LLMs) demonstrate potential in complex legal tasks like argument generation, yet their reliability remains a concern. Building upon pilot work assessing LLM…
Analyzing Images of Legal Documents: Toward Multi-Modal LLMs for Access to Justice
Hannes Westermann, Jaromir Savelka
Interacting with the legal system and the government requires the assembly and analysis of various pieces of information that can be spread across different (paper) documents, such…
Using LLMs to Discover Legal Factors
Morgan Gray, Jaromir Savelka, Wesley Oliver +1
Factors are a foundational component of legal analysis and computational models of legal reasoning. These factor-based representations enable lawyers, judges, and AI and Law resear…
Understanding the Role of Temperature in Diverse Question Generation by GPT-4
Arav Agarwal, Karthik Mittal, Aidan Doyle +5
We conduct a preliminary study of the effect of GPT's temperature parameter on the diversity of GPT4-generated questions. We find that using higher temperature values leads to sign…
From Text to Structure: Using Large Language Models to Support the Development of Legal Expert Systems
Samyar Janatian, Hannes Westermann, Jinzhe Tan +2
Encoding legislative text in a formal representation is an important prerequisite to different tasks in the field of AI & Law. For example, rule-based expert systems focused on leg…
LLMediator: GPT-4 Assisted Online Dispute Resolution
Hannes Westermann, Jaromir Savelka, Karim Benyekhlef
In this article, we introduce LLMediator, an experimental platform designed to enhance online dispute resolution (ODR) by utilizing capabilities of state-of-the-art large language…
Can GPT-4 Support Analysis of Textual Data in Tasks Requiring Highly Specialized Domain Expertise?
Jaromir Savelka, Kevin D. Ashley, Morgan A Gray +2
We evaluated the capability of generative pre-trained transformers~(GPT-4) in analysis of textual data in tasks that require highly specialized domain expertise. Specifically, we f…
Explaining Legal Concepts with Augmented Large Language Models (GPT-4)
Jaromir Savelka, Kevin D. Ashley, Morgan A. Gray +2
Interpreting the meaning of legal open-textured terms is a key task of legal professionals. An important source for this interpretation is how the term was applied in previous cour…
Unlocking Practical Applications in Legal Domain: Evaluation of GPT for Zero-Shot Semantic Annotation of Legal Texts
Jaromir Savelka
We evaluated the capability of a state-of-the-art generative pre-trained transformer (GPT) model to perform semantic annotation of short text snippets (one to few sentences) coming…
Large Language Models (GPT) Struggle to Answer Multiple-Choice Questions about Code
Jaromir Savelka, Arav Agarwal, Christopher Bogart +1
We analyzed effectiveness of three generative pre-trained transformer (GPT) models in answering multiple-choice question (MCQ) assessments, often involving short snippets of code,…
Toward an Intelligent Tutoring System for Argument Mining in Legal Texts
Hannes Westermann, Jaromir Savelka, Vern R. Walker +2
We propose an adaptive environment (CABINET) to support caselaw analysis (identifying key argument elements) based on a novel cognitive computing framework that carefully matches v…
Sentence Embeddings and High-speed Similarity Search for Fast Computer Assisted Annotation of Legal Documents
Hannes Westermann, Jaromir Savelka, Vern R. Walker +2
Human-performed annotation of sentences in legal documents is an important prerequisite to many machine learning based systems supporting legal tasks. Typically, the annotation is…
Lex Rosetta: Transfer of Predictive Models Across Languages, Jurisdictions, and Legal Domains
Jaromir Savelka, Hannes Westermann, Karim Benyekhlef +15
In this paper, we examine the use of multi-lingual sentence embeddings to transfer predictive models for functional segmentation of adjudicatory decisions across jurisdictions, leg…
Cross-Domain Generalization and Knowledge Transfer in Transformers Trained on Legal Data
Jaromir Savelka, Hannes Westermann, Karim Benyekhlef
We analyze the ability of pre-trained language models to transfer knowledge among datasets annotated with different type systems and to generalize beyond the domain and dataset the…
Discovering Explanatory Sentences in Legal Case Decisions Using Pre-trained Language Models
Jaromir Savelka, Kevin D. Ashley
Legal texts routinely use concepts that are difficult to understand. Lawyers elaborate on the meaning of such concepts by, among other things, carefully investigating how have they…
Citation Data of Czech Apex Courts
Jakub Harašta, Tereza Novotná, Jaromír Šavelka
In this paper, we introduce the citation data of the Czech apex courts (Supreme Court, Supreme Administrative Court and Constitutional Court). This dataset was automatically extrac…