7 citations · 10 across the 20 of their papers we have counts for
20 papers
DLawBench: Evaluating LLMs Through Multi-Turn Legal Consultation
Li Zhang, Yuzhen Shi, Yiran Hu +15
Lawyer-client consultation is a critical starting point for legal services. Effective legal assistance hinges on eliciting sufficient and truthful information from clients in order…
Agents' Last Exam
Yiyou Sun, Xinyang Han, Weichen Zhang +378
Recent AI systems have achieved strong results on a wide range of benchmarks, yet these gains have not translated into economically meaningful deployment across many professional d…
LexRubric: A Rubric-Guided Diagnostic Benchmark for Open-Ended Legal Tasks
Yifan Chen, Haitao Li, Yiran Hu +6
As large language models (LLMs) are increasingly applied to real-world legal tasks, evaluating the reliability of their open-ended legal responses has become essential. These tasks…
PLawBench: A Rubric-Based Benchmark for Evaluating LLMs in Real-World Legal Practice
Yuzhen Shi, Huanghai Liu, Yiran Hu +27
As large language models (LLMs) are increasingly applied to legal domain-specific tasks, evaluating their ability to perform legal work in real-world settings has become essential.…
Evaluation of Large Language Models in Legal Applications: Challenges, Methods, and Future Directions
Yiran Hu, Huanghai Liu, Chong Wang +15
Large language models (LLMs) are being increasingly integrated into legal applications, including judicial decision support, legal practice assistance, and public-facing legal serv…
AppellateGen: A Benchmark for Appellate Legal Judgment Generation
Hongkun Yang, Lionel Z. Wang, Wei Fan +10
Legal judgment generation is a critical task in legal intelligence. However, existing research in legal judgment generation has predominantly focused on first-instance trials, rely…
JustEva: A Toolkit to Evaluate LLM Fairness in Legal Knowledge Inference
Zongyue Xue, Siyuan Zheng, Shaochun Wang +8
The integration of Large Language Models (LLMs) into legal practice raises pressing concerns about judicial fairness, particularly due to the nature of their "black-box" processes.…
TENET: One Step Toward Test-Driven Development for Repository-Level Code Generation
Yiran Hu, Shanchao Liang, Nan Jiang +2
Test-Driven Development (TDD) is a widely adopted practice that requires developers to create and execute tests alongside implementation. With recent advances in Large Language Mod…
LLMs on Trial: Evaluating Judicial Fairness for Large Language Models
Yiran Hu, Zongyue Xue, Haitao Li +10
Large Language Models (LLMs) are increasingly used in high-stakes fields where their decisions impact rights and equity. However, LLMs' judicial fairness and implications for socia…
Legal Rule Induction: Towards Generalizable Principle Discovery from Analogous Judicial Precedents
Wei Fan, Tianshi Zheng, Yiran Hu +7
Legal rules encompass not only codified statutes but also implicit adjudicatory principles derived from precedents that contain discretionary norms, social morality, and policy. Wh…
JuDGE: Benchmarking Judgment Document Generation for Chinese Legal System
Weihang Su, Baoqing Yue, Qingyao Ai +6
This paper introduces JuDGE (Judgment Document Generation Evaluation), a novel benchmark for evaluating the performance of judgment document generation in the Chinese legal system.…
J&H: Evaluating the Robustness of Large Language Models Under Knowledge-Injection Attacks in Legal Domain
Yiran Hu, Huanghai Liu, Qingjing Chen +5
As the scale and capabilities of Large Language Models (LLMs) increase, their applications in knowledge-intensive fields such as legal domain have garnered widespread attention. Ho…
CaseGen: A Benchmark for Multi-Stage Legal Case Documents Generation
Haitao Li, Jiaying Ye, Yiran Hu +8
Legal case documents play a critical role in judicial proceedings. As the number of cases continues to rise, the reliance on manual drafting of legal case documents is facing incre…
JUREX-4E: Juridical Expert-Annotated Four-Element Knowledge Base for Legal Reasoning
Huanghai Liu, Quzhe Huang, Qingjing Chen +5
In recent years, Large Language Models (LLMs) have been widely applied to legal tasks. To enhance their understanding of legal texts and improve reasoning accuracy, a promising app…
LegalAgentBench: Evaluating LLM Agents in Legal Domain
Haitao Li, Junjie Chen, Jingli Yang +10
With the increasing intelligence and autonomy of LLM agents, their potential applications in the legal domain are becoming increasingly apparent. However, existing general-domain b…
Legal Fact Prediction: The Missing Piece in Legal Judgment Prediction
Junkai Liu, Yujie Tong, Hui Huang +7
Legal judgment prediction (LJP), which enables litigants and their lawyers to forecast judgment outcomes and refine litigation strategies, has emerged as a crucial legal NLP task.…
LeKUBE: A Legal Knowledge Update BEnchmark
Changyue Wang, Weihang Su, Hu Yiran +6
Recent advances in Large Language Models (LLMs) have significantly shaped the applications of AI in multiple fields, including the studies of legal intelligence. Trained on extensi…
STARD: A Chinese Statute Retrieval Dataset with Real Queries Issued by Non-professionals
Weihang Su, Yiran Hu, Anzhe Xie +6
Statute retrieval aims to find relevant statutory articles for specific queries. This process is the basis of a wide range of legal applications such as legal advice, automated jud…
MUSER: A Multi-View Similar Case Retrieval Dataset
Qingquan Li, Yiran Hu, Feng Yao +4
Similar case retrieval (SCR) is a representative legal AI application that plays a pivotal role in promoting judicial fairness. However, existing SCR datasets only focus on the fac…
LEEC: A Legal Element Extraction Dataset with an Extensive Domain-Specific Label System
Xue Zongyue, Liu Huanghai, Hu Yiran +4
As a pivotal task in natural language processing, element extraction has gained significance in the legal domain. Extracting legal elements from judicial documents helps enhance in…