416 citations · 3k across the 127 of their papers we have counts for
19 papers · 1 filter
LayoutNUWA: Revealing the Hidden Layout Expertise of Large Language Models
Zecheng Tang, Chenfei Wu, Juntao Li +1
Graphic layout generation, a growing research field, plays a significant role in user engagement and information perception. Existing methods primarily treat layout generation as a…
ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving
Zhibin Gou, Zhihong Shao, Yeyun Gong +5
Large language models have made significant progress in various language tasks, yet they still struggle with complex mathematics. In this paper, we propose ToRA a series of Tool-in…
Enhancing Large Language Models in Coding Through Multi-Perspective Self-Consistency
Baizhou Huang, Shuai Lu, Weizhu Chen +2
Large language models (LLMs) have exhibited remarkable ability in code generation. However, generating the correct solution in a single attempt still remains a challenge. Prior wor…
Constructing Multilingual Code Search Dataset Using Neural Machine Translation
Ryo Sekizawa, Nan Duan, Shuai Lu +1
Code search is a task to find programming codes that semantically match the given natural language queries. Even though some of the existing datasets for this task are multilingual…
LongCoder: A Long-Range Pre-trained Language Model for Code Completion
Daya Guo, Canwen Xu, Nan Duan +2
In this paper, we introduce a new task for code completion that focuses on handling long code input and propose a sparse Transformer model, called LongCoder, to address this task.…
CMMLU: Measuring massive multitask language understanding in Chinese
Haonan Li, Yixuan Zhang, Fajri Koto +5
As the capabilities of large language models (LLMs) continue to advance, evaluating their performance becomes increasingly crucial and challenging. This paper aims to bridge this g…
Code Execution with Pre-trained Language Models
Chenxiao Liu, Shuai Lu, Weizhu Chen +5
Code execution is a fundamental aspect of programming language semantics that reflects the exact behavior of the code. However, most pre-trained models for code intelligence ignore…
Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergy
Zhihong Shao, Yeyun Gong, Yelong Shen +3
Large language models are powerful text processors and reasoners, but are still subject to limitations including outdated knowledge and hallucinations, which necessitates connectin…
Allies: Prompting Large Language Model with Beam Search
Hao Sun, Xiao Liu, Yeyun Gong +4
With the advance of large language models (LLMs), the research field of LLM applications becomes more and more popular and the idea of constructing pipelines to accomplish complex…
Query Rewriting for Retrieval-Augmented Large Language Models
Xinbei Ma, Yeyun Gong, Pengcheng He +2
Large Language Models (LLMs) play powerful, black-box readers in the retrieve-then-read pipeline, making remarkable progress in knowledge-intensive tasks. This work introduces a ne…
Machine-Created Universal Language for Cross-lingual Transfer
Yaobo Liang, Quanzhi Zhu, Junhe Zhao +1
There are two primary approaches to addressing cross-lingual transfer: multilingual pre-training, which implicitly aligns the hidden representations of various languages, and trans…
CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing
Zhibin Gou, Zhihong Shao, Yeyun Gong +4
Recent developments in large language models (LLMs) have been impressive. However, these models sometimes show inconsistencies and problematic behavior, such as hallucinating facts…
AR-Diffusion: Auto-Regressive Diffusion Model for Text Generation
Tong Wu, Zhihao Fan, Xiao Liu +9
Diffusion models have gained significant attention in the realm of image generation due to their exceptional performance. Their success has been recently expanded to text generatio…
PROM: A Phrase-level Copying Mechanism with Pre-training for Abstractive Summarization
Xinbei Ma, Yeyun Gong, Pengcheng He +2
Based on the remarkable achievements of pre-trained language models in abstractive summarization, the copying mechanism has proved helpful by improving the factuality, stability, a…
Enhancing Chain-of-Thoughts Prompting with Iterative Bootstrapping in Large Language Models
Jiashuo Sun, Yi Luo, Yeyun Gong +4
Large language models (LLMs) can achieve highly effective performance on various reasoning tasks by incorporating step-by-step chain-of-thought (CoT) prompting as demonstrations. H…
AGIEval: A Human-Centric Benchmark for Evaluating Foundation Models
Wanjun Zhong, Ruixiang Cui, Yiduo Guo +6
Evaluating the general abilities of foundation models to tackle human-level tasks is a vital aspect of their development and application in the pursuit of Artificial General Intell…
AnnoLLM: Making Large Language Models to Be Better Crowdsourced Annotators
Xingwei He, Zhenghao Lin, Yeyun Gong +7
Many natural language processing (NLP) tasks rely on labeled data to train machine learning models with high performance. However, data annotation is time-consuming and expensive,…
Modeling Sequential Sentence Relation to Improve Cross-lingual Dense Retrieval
Shunyu Zhang, Yaobo Liang, Ming Gong +2
Recently multi-lingual pre-trained language models (PLM) such as mBERT and XLM-R have achieved impressive strides in cross-lingual dense retrieval. Despite its successes, they are…
Synthetic Prompting: Generating Chain-of-Thought Demonstrations for Large Language Models
Zhihong Shao, Yeyun Gong, Yelong Shen +3
Large language models can perform various reasoning tasks by using chain-of-thought prompting, which guides them to find answers through step-by-step demonstrations. However, the q…