38 citations · 67 across the 44 of their papers we have counts for
19 papers · 1 filter
GenRubric: Self-Evolving Rubric Generation for Scalable LLM Evaluation
Yifan Chen, Haitao Li, Qingyao Ai +4
Large language models are increasingly used as scalable evaluators for open-ended tasks. However, many LLM judges derive query-specific criteria during scoring, leaving the evaluat…
Civil Court Simulation with Large Language Models
Yifan Chen, Haitao Li, Kaiyuan Zhang +3
Court simulation bridges legal education and judicial practice, yet human-based simulations are costly and difficult to scale. Large language models (LLMs) offer a scalable alterna…
LexRubric: A Rubric-Guided Diagnostic Benchmark for Open-Ended Legal Tasks
Yifan Chen, Haitao Li, Yiran Hu +6
As large language models (LLMs) are increasingly applied to real-world legal tasks, evaluating the reliability of their open-ended legal responses has become essential. These tasks…
Benchmarking LLM-as-a-Judge for Long-Form Output Evaluation
Junjie Chen, Yuxi Dong, Haitao Li +5
As large language models (LLMs) are increasingly used for long-form generation, reliably evaluating long-form outputs has become a critical challenge. LLM-as-a-judge offers a scala…
ATACompressor: Adaptive Task-Aware Compression for Efficient Long-Context Processing in LLMs
Xuancheng Li, Haitao Li, Yujia Zhou +2
Long-context inputs in large language models (LLMs) often suffer from the "lost in the middle" problem, where critical information becomes diluted or ignored due to excessive lengt…
LegalOne: A Family of Foundation Models for Reliable Legal Reasoning
Haitao Li, Yifan Chen, Shuo Miao +13
While Large Language Models (LLMs) have demonstrated impressive general capabilities, their direct application in the legal domain is often hindered by a lack of precise domain kno…
LLMs on Trial: Evaluating Judicial Fairness for Large Language Models
Yiran Hu, Zongyue Xue, Haitao Li +10
Large Language Models (LLMs) are increasingly used in high-stakes fields where their decisions impact rights and equity. However, LLMs' judicial fairness and implications for socia…
Overview of the NTCIR-18 Automatic Evaluation of LLMs (AEOLLM) Task
Junjie Chen, Haitao Li, Zhumin Chu +2
In this paper, we provide an overview of the NTCIR-18 Automatic Evaluation of LLMs (AEOLLM) task. As large language models (LLMs) grow popular in both academia and industry, how to…
LexRAG: Benchmarking Retrieval-Augmented Generation in Multi-Turn Legal Consultation Conversation
Haitao Li, Yifan Chen, Yiran Hu +7
Retrieval-augmented generation (RAG) has proven highly effective in improving large language models (LLMs) across various domains. However, there is no benchmark specifically desig…
CaseGen: A Benchmark for Multi-Stage Legal Case Documents Generation
Haitao Li, Jiaying Ye, Yiran Hu +8
Legal case documents play a critical role in judicial proceedings. As the number of cases continues to rise, the reliance on manual drafting of legal case documents is facing incre…
LegalAgentBench: Evaluating LLM Agents in Legal Domain
Haitao Li, Junjie Chen, Jingli Yang +10
With the increasing intelligence and autonomy of LLM agents, their potential applications in the legal domain are becoming increasingly apparent. However, existing general-domain b…
LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods
Haitao Li, Qian Dong, Junjie Chen +5
The rapid advancement of Large Language Models (LLMs) has driven their expanding application across various fields. One of the most promising applications is their role as evaluato…
CalibraEval: Calibrating Prediction Distribution to Mitigate Selection Bias in LLMs-as-Judges
Haitao Li, Junjie Chen, Qingyao Ai +4
The use of large language models (LLMs) as automated evaluation tools to assess the quality of generated natural language, known as LLMs-as-Judges, has demonstrated promising capab…
Auto-PRE: An Automatic and Cost-Efficient Peer-Review Framework for Language Generation Evaluation
Junjie Chen, Weihang Su, Zhumin Chu +9
The rapid development of large language models (LLMs) has highlighted the need for efficient and reliable methods to evaluate their performance. Traditional evaluation methods ofte…
LexEval: A Comprehensive Chinese Legal Benchmark for Evaluating Large Language Models
Haitao Li, You Chen, Qingyao Ai +3
Large language models (LLMs) have made significant progress in natural language processing tasks and demonstrate considerable potential in the legal domain. However, legal applicat…
BLADE: Enhancing Black-box Large Language Models with Small Domain-Specific Models
Haitao Li, Qingyao Ai, Jia Chen +5
Large Language Models (LLMs) like ChatGPT and GPT-4 are versatile and capable of addressing a diverse range of tasks. However, general LLMs, which are developed on open-domain data…
Evaluation Ethics of LLMs in Legal Domain
Ruizhe Zhang, Haitao Li, Yueyue Wu +4
In recent years, the utilization of large language models for natural language dialogue has gained momentum, leading to their widespread adoption across various domains. However, t…
LeCaRDv2: A Large-Scale Chinese Legal Case Retrieval Dataset
Haitao Li, Yunqiu Shao, Yueyue Wu +3
As an important component of intelligent legal systems, legal case retrieval plays a critical role in ensuring judicial justice and fairness. However, the development of legal case…
THUIR@COLIEE 2023: More Parameters and Legal Knowledge for Legal Case Entailment
Haitao Li, Changyue Wang, Weihang Su +3
This paper describes the approach of the THUIR team at the COLIEE 2023 Legal Case Entailment task. This task requires the participant to identify a specific paragraph from a given…