5 papers
MetaXCR: Reinforcement-Based Meta-Transfer Learning for Cross-Lingual Commonsense Reasoning
Jie He, Yu Fu
Commonsense reasoning (CR) has been studied in many pieces of domain and has achieved great progress with the aid of large datasets. Unfortunately, most existing CR datasets are bu…
Evaluating Discourse Cohesion in Pre-trained Language Models
Jie He, Wanqiu Long, Deyi Xiong
Large pre-trained neural models have achieved remarkable success in natural language process (NLP), inspiring a growing body of research analyzing their ability from different aspe…
Tgea: An error-annotated dataset and benchmark tasks for text generation from pretrained language models
Jie He, Bo Peng, Yi Liao +2
In order to deeply understand the capability of pretrained language models in text generation and conduct a diagnostic evaluation, we propose TGEA, an error-annotated dataset with…
The Box is in the Pen: Evaluating Commonsense Reasoning in Neural Machine Translation
Jie He, Tao Wang, Deyi Xiong +1
Does neural machine translation yield translations that are congenial with common sense? In this paper, we present a test suite to evaluate the commonsense reasoning capability of…
MINTQA: A Multi-Hop Question Answering Benchmark for Evaluating LLMs on New and Tail Knowledge
Jie He, Nan Hu, Wanqiu Long +2
Large language models (LLMs) have demonstrated impressive capabilities in various reasoning tasks but face significant challenges with complex, knowledge-intensive multi-hop querie…