activity
20222026
most citedLLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods

38 citations · 103 across the 93 of their papers we have counts for

collaborators
Showing 2024Show all

32 papers · 1 filter

cs.IR2024

Unsupervised dense retrieval with conterfactual contrastive learning

Haitian Chen, Qingyao Ai, Xiao Wang +3

Efficiently retrieving a concise set of candidates from a large document corpus remains a pivotal challenge in Information Retrieval (IR). Neural retrieval models, particularly den…

cs.CL2024★ 1 cited

LegalAgentBench: Evaluating LLM Agents in Legal Domain

Haitao Li, Junjie Chen, Jingli Yang +10

With the increasing intelligence and autonomy of LLM agents, their potential applications in the legal domain are becoming increasingly apparent. However, existing general-domain b…

cs.CL2024★ 38 cited

LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods

Haitao Li, Qian Dong, Junjie Chen +5

The rapid advancement of Large Language Models (LLMs) has driven their expanding application across various fields. One of the most promising applications is their role as evaluato…

cs.CL2024

Knowledge Editing through Chain-of-Thought

Changyue Wang, Weihang Su, Qingyao Ai +2

Knowledge Editing is a technique that updates large language models (LLMs) with new information to maintain their world knowledge. This approach avoids the need to rebuild the mode…

cs.CL2024

CalibraEval: Calibrating Prediction Distribution to Mitigate Selection Bias in LLMs-as-Judges

Haitao Li, Junjie Chen, Qingyao Ai +4

The use of large language models (LLMs) as automated evaluation tools to assess the quality of generated natural language, known as LLMs-as-Judges, has demonstrated promising capab…

cs.CL2024★ 2 cited

Beyond Scalar Reward Model: Learning Generative Judge from Preference Data

Ziyi Ye, Xiangsheng Li, Qiuchi Li +5

Learning from preference feedback is a common practice for aligning large language models~(LLMs) with human value. Conventionally, preference data is learned and encoded into a sca…

cs.CL2024

Auto-PRE: An Automatic and Cost-Efficient Peer-Review Framework for Language Generation Evaluation

Junjie Chen, Weihang Su, Zhumin Chu +9

The rapid development of large language models (LLMs) has highlighted the need for efficient and reliable methods to evaluate their performance. Traditional evaluation methods ofte…

cs.CL2024★ 3 cited

LexEval: A Comprehensive Chinese Legal Benchmark for Evaluating Large Language Models

Haitao Li, You Chen, Qingyao Ai +3

Large language models (LLMs) have made significant progress in natural language processing tasks and demonstrate considerable potential in the legal domain. However, legal applicat…

cs.CL2024★ 3 cited

Mitigating Entity-Level Hallucination in Large Language Models

Weihang Su, Yichen Tang, Qingyao Ai +3

The emergence of Large Language Models (LLMs) has revolutionized how users access information, shifting from traditional search engines to direct question-and-answer interactions w…

cs.CL2024

LeKUBE: A Legal Knowledge Update BEnchmark

Changyue Wang, Weihang Su, Hu Yiran +6

Recent advances in Large Language Models (LLMs) have significantly shaped the applications of AI in multiple fields, including the studies of legal intelligence. Trained on extensi…

cs.CV2024

Prompt Refinement with Image Pivot for Text-to-Image Generation

Jingtao Zhan, Qingyao Ai, Yiqun Liu +5

For text-to-image generation, automatically refining user-provided natural language prompts into the keyword-enriched prompts favored by systems is essential for the user experienc…

cs.IR2024

STARD: A Chinese Statute Retrieval Dataset with Real Queries Issued by Non-professionals

Weihang Su, Yiran Hu, Anzhe Xie +6

Statute retrieval aims to find relevant statutory articles for specific queries. This process is the basis of a wide range of legal applications such as legal advice, automated jud…

cs.MM2024★ 4 cited

CrossPT-EEG: A Benchmark for Cross-Participant and Cross-Time Generalization of EEG-based Visual Decoding

Shuqi Zhu, Ziyi Ye, Qingyao Ai +1

Exploring brain activity in relation to visual perception provides insights into the biological representation of the world. While functional magnetic resonance imaging (fMRI) and…

cs.IR2024

EEG-SVRec: An EEG Dataset with User Multidimensional Affective Engagement Labels in Short Video Recommendation

Shaorun Zhang, Zhiyu He, Ziyi Ye +4

In recent years, short video platforms have gained widespread popularity, making the quality of video recommendations crucial for retaining users. Existing recommendation systems p…

cs.IR2024

Towards an In-Depth Comprehension of Case Relevance for Better Legal Retrieval

Haitao Li, You Chen, Zhekai Ge +4

Legal retrieval techniques play an important role in preserving the fairness and equality of the judicial system. As an annually well-known international competition, COLIEE aims t…

cs.CL2024

Capability-aware Prompt Reformulation Learning for Text-to-Image Generation

Jingtao Zhan, Qingyao Ai, Yiqun Liu +2

Text-to-image generation systems have emerged as revolutionary tools in the realm of artistic creation, offering unprecedented ease in transforming textual prompts into visual art.…

cs.IR2024

DELTA: Pre-train a Discriminative Encoder for Legal Case Retrieval via Structural Word Alignment

Haitao Li, Qingyao Ai, Xinyan Han +5

Recent research demonstrates the effectiveness of using pre-trained language models for legal case retrieval. Most of the existing works focus on improving the representation abili…

cs.CL2024★ 1 cited

BLADE: Enhancing Black-box Large Language Models with Small Domain-Specific Models

Haitao Li, Qingyao Ai, Jia Chen +5

Large Language Models (LLMs) like ChatGPT and GPT-4 are versatile and capable of addressing a diverse range of tasks. However, general LLMs, which are developed on open-domain data…

cs.IR2024★ 3 cited

Sequential Recommendation with Latent Relations based on Large Language Model

Shenghao Yang, Weizhi Ma, Peijie Sun +4

Sequential recommender systems predict items that may interest users by modeling their preferences based on historical interactions. Traditional sequential recommendation methods r…

cs.IR2024★ 2 cited

Common Sense Enhanced Knowledge-based Recommendation with Large Language Model

Shenghao Yang, Weizhi Ma, Peijie Sun +4

Knowledge-based recommendation models effectively alleviate the data sparsity issue leveraging the side information in the knowledge graph, and have achieved considerable performan…

cs.IR2024

A Situation-aware Enhancer for Personalized Recommendation

Jiayu Li, Peijie Sun, Chumeng Jiang +3

When users interact with Recommender Systems (RecSys), current situations, such as time, location, and environment, significantly influence their preferences. Situations serve as t…

cs.IR2024

Improving Legal Case Retrieval with Brain Signals

Ruizhe Zhang, Qingyao Ai, Ziyi Ye +3

The tasks of legal case retrieval have received growing attention from the IR community in the last decade. Relevance feedback techniques with implicit user feedback (e.g., clicks)…

cs.CL2024★ 1 cited

Evaluation Ethics of LLMs in Legal Domain

Ruizhe Zhang, Haitao Li, Yueyue Wu +4

In recent years, the utilization of large language models for natural language dialogue has gained momentum, leading to their widespread adoption across various domains. However, t…

cs.CL2024

Query Augmentation by Decoding Semantics from Brain Signals

Ziyi Ye, Jingtao Zhan, Qingyao Ai +4

Query augmentation is a crucial technique for refining semantically imprecise queries. Traditionally, query augmentation relies on extracting information from initially retrieved,…