7 papers · 1 filter
MINOS: A Multimodal Evaluation Model for Bidirectional Generation Between Image and Text
Junzhe Zhang, Huixuan Zhang, Xinyu Hu +4
Evaluation is important for multimodal generation tasks, while traditional multimodal evaluation metrics suffer from several limitations. With the rapid progress of MLLMs, there is…
LEDOM: Reverse Language Model
Xunjian Yin, Sitao Cheng, Yuxi Xie +6
Autoregressive language models are trained exclusively left-to-right. We explore the complementary factorization, training right-to-left at scale, and ask what reasoning patterns e…
CE-RM: A Pointwise Generative Reward Model Optimized via Two-Stage Rollout and Unified Criteria
Xinyu Hu, Yancheng He, Weixun Wang +6
Automatic evaluation is crucial yet challenging for open-ended natural language generation, especially when rule-based metrics are infeasible. Compared with traditional methods, th…
HAD: HAllucination Detection Language Models Based on a Comprehensive Hallucination Taxonomy
Fan Xu, Xinyu Hu, Zhenghan Yu +6
The increasing reliance on natural language generation (NLG) models, particularly large language models, has raised concerns about the reliability and accuracy of their outputs. A…
A Dual-Perspective NLG Meta-Evaluation Framework with Automatic Benchmark and Better Interpretability
Xinyu Hu, Mingqi Gao, Li Lin +2
In NLG meta-evaluation, evaluation metrics are typically assessed based on their consistency with humans. However, we identify some limitations in traditional NLG meta-evaluation a…
Analyzing and Evaluating Correlation Measures in NLG Meta-Evaluation
Mingqi Gao, Xinyu Hu, Li Lin +1
The correlation between NLG automatic evaluation metrics and human evaluation is often regarded as a critical criterion for assessing the capability of an evaluation metric. Howeve…