21 citations · 48 across the 24 of their papers we have counts for
26 papers
Can LLM Agents Stick to the Script? A Benchmark for Long-Horizon Consistency in Interactive Narratives
Yingpeng Ma, Jianhao Yan, Bei Shi +6
The rapid advancement of Large Language Models (LLMs) is revolutionizing AI for Games by enabling open-ended and fluid interactive storytelling. However, existing research has larg…
Learning to Reason under Off-Policy Guidance
Jianhao Yan, Yafu Li, Zican Hu +5
Recent advances in large reasoning models (LRMs) demonstrate that sophisticated behaviors such as multi-step reasoning and self-reflection can emerge via reinforcement learning wit…
A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyond
Xiaoye Qu, Yafu Li, Zhao-Chen Su +15
Recent Large Reasoning Models (LRMs), such as DeepSeek-R1 and OpenAI o1, have demonstrated strong performance gains by scaling up the length of Chain-of-Thought (CoT) reasoning dur…
RefuteBench 2.0 -- Agentic Benchmark for Dynamic Evaluation of LLM Responses to Refutation Instruction
Jianhao Yan, Yun Luo, Yue Zhang
In the multi-turn interaction schema, large language models (LLMs) can leverage user feedback to enhance the quality and relevance of their responses. However, evaluating an LLM's…
Benchmarking GPT-4 against Human Translators: A Comprehensive Evaluation Across Languages, Domains, and Expertise Levels
Jianhao Yan, Pingchuan Yan, Yulong Chen +3
This study presents a comprehensive evaluation of GPT-4's translation capabilities compared to human translators of varying expertise levels. Through systematic human evaluation us…
ELICIT: LLM Augmentation via External In-Context Capability
Futing Wang, Jianhao Yan, Yue Zhang +1
Enhancing the adaptive capabilities of large language models is a critical pursuit in both research and application. Traditional fine-tuning methods require substantial data and co…
Keys to Robust Edits: from Theoretical Insights to Practical Advances
Jianhao Yan, Futing Wang, Yun Luo +2
Large language models (LLMs) struggle with maintaining accurate knowledge due to conflicting/outdated parametric memories. While locate-and-edit methods address this, their relianc…
See What LLMs Cannot Answer: A Self-Challenge Framework for Uncovering LLM Weaknesses
Yulong Chen, Yang Liu, Jianhao Yan +6
The impressive performance of Large Language Models (LLMs) has consistently surpassed numerous human-designed benchmarks, presenting new challenges in assessing the shortcomings of…
GPT-4 vs. Human Translators: A Comprehensive Evaluation of Translation Quality Across Languages, Domains, and Expertise Levels
Jianhao Yan, Pingchuan Yan, Yulong Chen +3
This study comprehensively evaluates the translation quality of Large Language Models (LLMs), specifically GPT-4, against human translators of varying expertise levels across multi…
What Have We Achieved on Non-autoregressive Translation?
Yafu Li, Huajian Zhang, Jianhao Yan +2
Recent advances have made non-autoregressive (NAT) translation comparable to autoregressive methods (AT). However, their evaluation using BLEU has been shown to weakly correlate wi…
Potential and Challenges of Model Editing for Social Debiasing
Jianhao Yan, Futing Wang, Yafu Li +1
Large language models (LLMs) trained on vast corpora suffer from inevitable stereotype biases. Mitigating these biases with fine-tuning could be both costly and data-hungry. Model…
RefuteBench: Evaluating Refuting Instruction-Following for Large Language Models
Jianhao Yan, Yun Luo, Yue Zhang
The application scope of large language models (LLMs) is increasingly expanding. In practical use, users might provide feedback based on the model's output, hoping for a responsive…
Understanding In-Context Learning from Repetitions
Jianhao Yan, Jin Xu, Chiyu Song +3
This paper explores the elusive mechanism underpinning in-context learning in Large Language Models (LLMs). Our work provides a novel perspective by examining in-context learning v…
Revisiting Cross-Lingual Summarization: A Corpus-based Study and A New Benchmark with Improved Annotation
Yulong Chen, Huajian Zhang, Yijie Zhou +8
Most existing cross-lingual summarization (CLS) work constructs CLS corpora by simply and directly translating pre-annotated summaries from one language to another, which can conta…
Explicit Syntactic Guidance for Neural Text Generation
Yafu Li, Leyang Cui, Jianhao Yan +4
Most existing text generation models follow the sequence-to-sequence paradigm. Generative Grammar suggests that humans generate natural language texts by learning language grammar.…
Non-Autoregressive Document-Level Machine Translation
Guangsheng Bao, Zhiyang Teng, Hao Zhou +2
Non-autoregressive translation (NAT) models achieve comparable performance and superior speed compared to auto-regressive translation (AT) models in the context of sentence-level m…
DC-MBR: Distributional Cooling for Minimum Bayesian Risk Decoding
Jianhao Yan, Jin Xu, Fandong Meng +2
Minimum Bayesian Risk Decoding (MBR) emerges as a promising decoding algorithm in Neural Machine Translation. However, MBR performs poorly with label smoothing, which is surprising…
Probing Causes of Hallucinations in Neural Machine Translations
Jianhao Yan, Fandong Meng, Jie Zhou
Hallucination, one kind of pathological translations that bothers Neural Machine Translation, has recently drawn much attention. In simple terms, hallucinated translations are flue…
Learning to Break the Loop: Analyzing and Mitigating Repetitions for Neural Text Generation
Jin Xu, Xiaojiang Liu, Jianhao Yan +3
While large-scale neural language models, such as GPT2 and BART, have achieved impressive results on various text generation tasks, they tend to get stuck in undesirable sentence-l…
Digging Errors in NMT: Evaluating and Understanding Model Errors from Partial Hypothesis Space
Jianhao Yan, Chenming Wu, Fandong Meng +1
Solid evaluation of neural machine translation (NMT) is key to its understanding and improvement. Current evaluation of an NMT system is usually built upon a heuristic decoding alg…
Selective Knowledge Distillation for Neural Machine Translation
Fusheng Wang, Jianhao Yan, Fandong Meng +1
Neural Machine Translation (NMT) models achieve state-of-the-art performance on many translation benchmarks. As an active research field in NMT, knowledge distillation is widely ap…
Multi-Unit Transformers for Neural Machine Translation
Jianhao Yan, Fandong Meng, Jie Zhou
Transformer models achieve remarkable success in Neural Machine Translation. Many efforts have been devoted to deepening the Transformer by stacking several units (i.e., a combinat…
A Sentiment-Controllable Topic-to-Essay Generator with Topic Knowledge Graph
Lin Qiao, Jianhao Yan, Fandong Meng +2
Generating a vivid, novel, and diverse essay with only several given topic words is a challenging task of natural language generation. In previous work, there are two problems left…
WeChat Neural Machine Translation Systems for WMT20
Fandong Meng, Jianhao Yan, Yijin Liu +8
We participate in the WMT 2020 shared news translation task on Chinese to English. Our system is based on the Transformer (Vaswani et al., 2017a) with effective variants and the DT…