519 citations · 912 across the 14 of their papers we have counts for
12 papers · 1 filter
O1 Replication Journey -- Part 2: Surpassing O1-preview through Simple Distillation, Big Progress or Bitter Lesson?
Zhen Huang, Haoyang Zou, Xuefeng Li +7
This paper presents a critical examination of current approaches to replicating OpenAI's O1 model capabilities, with particular focus on the widespread but often undisclosed use of…
LLMCRIT: Teaching Large Language Models to Use Criteria
Weizhe Yuan, Pengfei Liu, Matthias Gallé
Humans follow criteria when they execute tasks, and these criteria are directly used to assess the quality of task completion. Therefore, having models learn to use criteria to pro…
The Critique of Critique
Shichao Sun, Junlong Li, Weizhe Yuan +3
Critique, as a natural language description for assessing the quality of model-generated content, has played a vital role in the training, evaluation, and refinement of LLMs. Howev…
Generative Judge for Evaluating Alignment
Junlong Li, Shichao Sun, Weizhe Yuan +3
The rapid development of Large Language Models (LLMs) has substantially expanded the range of tasks they can address. In the field of Natural Language Processing (NLP), researchers…
FacTool: Factuality Detection in Generative AI -- A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarios
I-Chun Chern, Steffi Chern, Shiqi Chen +6
The emergence of generative pre-trained models has facilitated the synthesis of high-quality text, but it has also posed challenges in identifying factual errors in the generated t…
System-Level Natural Language Feedback
Weizhe Yuan, Kyunghyun Cho, Jason Weston
Natural language (NL) feedback offers rich insights into user experience. While existing studies focus on an instance-level approach, where feedback is used to refine specific exam…
T5Score: Discriminative Fine-tuning of Generative Evaluation Metrics
Yiwei Qin, Weizhe Yuan, Graham Neubig +1
Modern embedding-based metrics for evaluation of generated text generally fall into one of two paradigms: discriminative metrics that are trained to directly predict which outputs…
reStructured Pre-training
Weizhe Yuan, Pengfei Liu
In this work, we try to decipher the internal connection of NLP technology development in the past decades, searching for essence, which rewards us with a (potential) new learning…
Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing
Pengfei Liu, Weizhe Yuan, Jinlan Fu +3
This paper surveys and organizes research works in a new paradigm in natural language processing, which we dub "prompt-based learning". Unlike traditional supervised learning, whic…
BARTScore: Evaluating Generated Text as Text Generation
Weizhe Yuan, Graham Neubig, Pengfei Liu
A wide variety of NLP applications, such as machine translation, summarization, and dialog, involve text generation. One major challenge for these applications is how to evaluate w…
ExplainaBoard: An Explainable Leaderboard for NLP
Pengfei Liu, Jinlan Fu, Yang Xiao +7
With the rapid development of NLP research, leaderboards have emerged as one tool to track the performance of various systems on various NLP tasks. They are effective in this goal…
Can We Automate Scientific Reviewing?
Weizhe Yuan, Pengfei Liu, Graham Neubig
The rapid development of science and technology has been accompanied by an exponential growth in peer-reviewed scientific publications. At the same time, the review of each paper i…