33 citations · 54 across the 18 of their papers we have counts for
19 papers
CE-RM: A Pointwise Generative Reward Model Optimized via Two-Stage Rollout and Unified Criteria
Xinyu Hu, Yancheng He, Weixun Wang +6
Automatic evaluation is crucial yet challenging for open-ended natural language generation, especially when rule-based metrics are infeasible. Compared with traditional methods, th…
SCOPE: Intrinsic Semantic Space Control for Mitigating Copyright Infringement in LLMs
Zhenliang Zhang, Xinyu Hu, Xiaojun Wan
Large language models sometimes inadvertently reproduce passages that are copyrighted, exposing downstream applications to legal risk. Most existing studies for inference-time defe…
LoaQ: Layer-wise Output Approximation Quantization
Li Lin, Xiaojun Wan
A natural and intuitive idea in model quantization is to approximate each component's quantized output to match its original. Motivated by this idea, most layer-wise post-training…
LEDOM: Reverse Language Model
Xunjian Yin, Sitao Cheng, Yuxi Xie +6
Autoregressive language models are trained exclusively left-to-right. We explore the complementary factorization, training right-to-left at scale, and ask what reasoning patterns e…
NeUQI: Near-Optimal Uniform Quantization Parameter Initialization for Low-Bit LLMs
Li Lin, Xinyu Hu, Xiaojun Wan
Large language models (LLMs) achieve impressive performance across domains but face significant challenges when deployed on consumer-grade GPUs or personal devices such as laptops,…
CFunModel: A "Funny" Language Model Capable of Chinese Humor Generation and Processing
Zhenghan Yu, Xinyu Hu, Xiaojun Wan
Humor plays a significant role in daily language communication. With the rapid development of large language models (LLMs), natural language processing has made significant strides…
Exploring the Multilingual NLG Evaluation Abilities of LLM-Based Evaluators
Jiayi Chang, Mingqi Gao, Xinyu Hu +1
Previous research has shown that LLMs have potential in multilingual NLG evaluation tasks. However, existing research has not fully explored the differences in the evaluation capab…
Aspect-Guided Multi-Level Perturbation Analysis of Large Language Models in Automated Peer Review
Jiatao Li, Yanheng Li, Xinyu Hu +2
We propose an aspect-guided, multi-level perturbation framework to evaluate the robustness of Large Language Models (LLMs) in automated peer review. Our framework explores perturba…
Re-evaluating Automatic LLM System Ranking for Alignment with Human Preference
Mingqi Gao, Yixin Liu, Xinyu Hu +3
Evaluating and ranking the capabilities of different LLMs is crucial for understanding their performance and alignment with human preferences. Due to the high cost and time-consumi…
Who Writes What: Unveiling the Impact of Author Roles on AI-generated Text Detection
Jiatao Li, Xiaojun Wan
The rise of Large Language Models (LLMs) necessitates accurate AI-generated text detection. However, current approaches largely overlook the influence of author characteristics. We…
A Dual-Perspective NLG Meta-Evaluation Framework with Automatic Benchmark and Better Interpretability
Xinyu Hu, Mingqi Gao, Li Lin +2
In NLG meta-evaluation, evaluation metrics are typically assessed based on their consistency with humans. However, we identify some limitations in traditional NLG meta-evaluation a…
Analyzing and Evaluating Correlation Measures in NLG Meta-Evaluation
Mingqi Gao, Xinyu Hu, Li Lin +1
The correlation between NLG automatic evaluation metrics and human evaluation is often regarded as a critical criterion for assessing the capability of an evaluation metric. Howeve…
Better than Random: Reliable NLG Human Evaluation with Constrained Active Sampling
Jie Ruan, Xiao Pu, Mingqi Gao +2
Human evaluation is viewed as a reliable evaluation method for NLG which is expensive and time-consuming. To save labor and costs, researchers usually perform human evaluation on a…
Defining and Detecting Vulnerability in Human Evaluation Guidelines: A Preliminary Study Towards Reliable NLG Evaluation
Jie Ruan, Wenqing Wang, Xiaojun Wan
Human evaluation serves as the gold standard for assessing the quality of Natural Language Generation (NLG) systems. Nevertheless, the evaluation guideline, as a pivotal element en…
Themis: A Reference-free NLG Evaluation Language Model with Flexibility and Interpretability
Xinyu Hu, Li Lin, Mingqi Gao +2
The evaluation of natural language generation (NLG) tasks is a significant and longstanding research area. With the recent emergence of powerful large language models (LLMs), some…
Are LLM-based Evaluators Confusing NLG Quality Criteria?
Xinyu Hu, Mingqi Gao, Sen Hu +4
Some prior work has shown that LLMs perform well in NLG evaluation for different tasks. However, we discover that LLMs seem to confuse different evaluation criteria, which reduces…
LLM-based NLG Evaluation: Current Status and Challenges
Mingqi Gao, Xinyu Hu, Jie Ruan +2
Evaluating natural language generation (NLG) is a vital but challenging problem in natural language processing. Traditional evaluation metrics mainly capturing content (e.g. n-gram…
OpinSummEval: Revisiting Automated Evaluation for Opinion Summarization
Yuchen Shen, Xiaojun Wan
Opinion summarization sets itself apart from other types of summarization tasks due to its distinctive focus on aspects and sentiments. Although certain automated evaluation method…
Summarization is (Almost) Dead
Xiao Pu, Mingqi Gao, Xiaojun Wan
How well can large language models (LLMs) generate summaries? We develop new datasets and conduct human evaluation experiments to evaluate the zero-shot generation capability of LL…