most citedA Survey on Evaluation of Large Language Models

202 citations · 310 across the 24 of their papers we have counts for

collaborators

24 papers

cs.CL2023★ 4 cited

A Survey of Text Watermarking in the Era of Large Language Models

Aiwei Liu, Leyi Pan, Yijian Lu +7

Text watermarking algorithms are crucial for protecting the copyright of textual content. Historically, their capabilities and application scenarios were limited. However, recent a…

cs.CL2023★ 2 cited

Prompt Me Up: Unleashing the Power of Alignments for Multimodal Entity and Relation Extraction

Xuming Hu, Junzhe Chen, Aiwei Liu +3

How can we better extract entities and relations from text? Using multimodal extraction with images and text obtains more signals for entities and relations, and aligns them throug…

cs.IR2023

AE-smnsMLC: Multi-Label Classification with Semantic Matching and Negative Label Sampling for Product Attribute Value Extraction

Zhongfen Deng, Wei-Te Chen, Lei Chen +1

Product attribute value extraction plays an important role for many real-world applications in e-Commerce such as product search and recommendation. Previous methods treat it as a…

cs.CL2023★ 9 cited

Do Large Language Models Know about Facts?

Xuming Hu, Junzhe Chen, Xiaochuan Li +4

Large language models (LLMs) have recently driven striking performance improvements across a range of natural language processing tasks. The factual knowledge acquired during pretr…

cs.CL2023

Enhancing Cross-lingual Transfer via Phonemic Transcription Integration

Hoang H. Nguyen, Chenwei Zhang, Tao Zhang +2

Previous cross-lingual transfer methods are restricted to orthographic representation learning via textual scripts. This limitation hampers cross-lingual transfer and is biased tow…

cs.CL2023★ 5 cited

An Unforgeable Publicly Verifiable Watermark for Large Language Models

Aiwei Liu, Leyi Pan, Xuming Hu +4

Recently, text watermarking algorithms for large language models (LLMs) have been proposed to mitigate the potential harms of text generated by LLMs, including fake news and copyri…

cs.CL2023★ 202 cited

A Survey on Evaluation of Large Language Models

Yupeng Chang, Xu Wang, Jindong Wang +13

Large language models (LLMs) are gaining increasing popularity in both academia and industry, owing to their unprecedented performance in various applications. As LLMs continue to…

cs.IR2023★ 1 cited

Multi-task Item-attribute Graph Pre-training for Strict Cold-start Item Recommendation

Yuwei Cao, Liangwei Yang, Chen Wang +4

Recommendation systems suffer in the strict cold-start (SCS) scenario, where the user-item interactions are entirely unavailable. The ID-based approaches completely fail to work. C…

cs.CL2023★ 15 cited

TrustGPT: A Benchmark for Trustworthy and Responsible Large Language Models

Yue Huang, Qihui Zhang, Philip S. Y +1

Large Language Models (LLMs) such as ChatGPT, have gained significant attention due to their impressive natural language processing capabilities. It is crucial to prioritize human-…

cs.CL2023

GDA: Generative Data Augmentation Techniques for Relation Extraction Tasks

Xuming Hu, Aiwei Liu, Zeqi Tan +4

Relation extraction (RE) tasks show promising performance in extracting relations from two entities mentioned in sentences, given sufficient annotations available during training.…

cs.IR2023★ 2 cited

Sequential Recommendation with Controllable Diversification: Representation Degeneration and Diversity

Ziwei Fan, Zhiwei Liu, Hao Peng +1

Sequential recommendation (SR) models the dynamic user preferences and generates the next-item prediction as the affinity between the sequence and items, in a joint latent space wi…

cs.CL2023

Multimodal Relation Extraction with Cross-Modal Retrieval and Synthesis

Xuming Hu, Zhijiang Guo, Zhiyang Teng +2

Multimodal relation extraction (MRE) is the task of identifying the semantic relationships between two entities based on the context of the sentence image pair. Existing retrieval-…

cs.CL2023

Enhancing Cross-lingual Natural Language Inference by Soft Prompting with Multilingual Verbalizer

Shuang Li, Xuming Hu, Aiwei Liu +4

Cross-lingual natural language inference is a fundamental problem in cross-lingual language understanding. Many recent works have used prompt learning to address the lack of annota…

cs.IR2023

Zero-shot Item-based Recommendation via Multi-task Product Knowledge Graph Pre-Training

Ziwei Fan, Zhiwei Liu, Shelby Heinecke +4

Existing recommender systems face difficulties with zero-shot items, i.e. items that have no historical interactions with users during the training stage. Though recent works extra…

cs.CL2023★ 1 cited

Read it Twice: Towards Faithfully Interpretable Fact Verification by Revisiting Evidence

Xuming Hu, Zhaochen Hong, Zhijiang Guo +2

Real-world fact verification task aims to verify the factuality of a claim by retrieving evidence from the source document. The quality of the retrieved evidence plays an important…

cs.CL2023★ 1 cited

Think Rationally about What You See: Continuous Rationale Extraction for Relation Extraction

Xuming Hu, Zhaochen Hong, Chenwei Zhang +2

Relation extraction (RE) aims to extract potential relations according to the context of two entities, thus, deriving rational contexts from sentences plays an important role. Prev…

cs.CL2023★ 2 cited

Give Me More Details: Improving Fact-Checking with Latent Retrieval

Xuming Hu, Junzhe Chen, Zhijiang Guo +1

Evidence plays a crucial role in automated fact-checking. When verifying real-world claims, existing fact-checking systems either assume the evidence sentences are given or use the…

cs.LG2023★ 1 cited

Conditional Denoising Diffusion for Sequential Recommendation

Yu Wang, Zhiwei Liu, Liangwei Yang +1

Generative models have attracted significant interest due to their ability to handle uncertainty by learning the inherent data distributions. However, two prominent generative mode…

cs.CL2023★ 61 cited

A comprehensive evaluation of ChatGPT's zero-shot Text-to-SQL capability

Aiwei Liu, Xuming Hu, Lijie Wen +1

This paper presents the first comprehensive analysis of ChatGPT's Text-to-SQL ability. Given the recent emergence of large-scale conversational language model ChatGPT and its impre…

cs.LG2023★ 1 cited

Provable Unrestricted Adversarial Training without Compromise with Generalizability

Lilin Zhang, Ning Yang, Yanchao Sun +1

Adversarial training (AT) is widely considered as the most promising strategy to defend against adversarial attacks and has drawn increasing interest from researchers. However, the…

cs.CL2022★ 1 cited

Learning to Select from Multiple Options

Jiangshu Du, Wenpeng Yin, Congying Xia +1

Many NLP tasks can be regarded as a selection problem from a set of options, such as classification tasks, multi-choice question answering, etc. Textual entailment (TE) has been sh…

cs.IR2022

Mitigating Frequency Bias in Next-Basket Recommendation via Deconfounders

Xiaohan Li, Zheng Liu, Luyi Ma +4

Recent studies on Next-basket Recommendation (NBR) have achieved much progress by leveraging Personalized Item Frequency (PIF) as one of the main features, which measures the frequ…

cs.CL2022

Continuous Prompt Tuning Based Textual Entailment Model for E-commerce Entity Typing

Yibo Wang, Congying Xia, Guan Wang +1

The explosion of e-commerce has caused the need for processing and analysis of product titles, like entity typing in product titles. However, the rapid activity in e-commerce has l…

cs.CL2022★ 2 cited

Entity-to-Text based Data Augmentation for various Named Entity Recognition Tasks

Xuming Hu, Yong Jiang, Aiwei Liu +5

Data augmentation techniques have been used to alleviate the problem of scarce labeled data in various NER tasks (flat, nested, and discontinuous NER tasks). Existing augmentation…