257 citations · 519 across the 28 of their papers we have counts for
21 papers · 1 filter
OpenCompass: A Universal Evaluation Platform for Large Language Models
Maosong Cao, Kai Chen, Haodong Duan +27
In recent years, the field of artificial intelligence has undergone a paradigm shift from task-specific small-scale models to general-purpose large language models (LLMs). With the…
Symbol-LLM: Towards Foundational Symbol-centric Interface For Large Language Models
Fangzhi Xu, Zhiyong Wu, Qiushi Sun +6
Although Large Language Models (LLMs) demonstrate remarkable ability in processing and generating human-like text, they do have limitations when it comes to comprehending and expre…
How Vocabulary Sharing Facilitates Multilingualism in LLaMA?
Fei Yuan, Shuai Yuan, Zhiyong Wu +1
Large Language Models (LLMs), often show strong performance on English tasks, while exhibiting limitations on other languages. What is an LLM's multilingual capability when it is t…
EMO: Earth Mover Distance Optimization for Auto-Regressive Language Modeling
Siyu Ren, Zhiyong Wu, Kenny Q. Zhu
Neural language models are probabilistic models of human text. They are predominantly trained using maximum likelihood estimation (MLE), which is equivalent to minimizing the forwa…
Can We Edit Factual Knowledge by In-Context Learning?
Ce Zheng, Lei Li, Qingxiu Dong +4
Previous studies have shown that large language models (LLMs) like GPTs store massive factual knowledge in their parameters. However, the stored knowledge could be false or out-dat…
OpenICL: An Open-Source Framework for In-context Learning
Zhenyu Wu, YaoXiang Wang, Jiacheng Ye +4
In recent years, In-context Learning (ICL) has gained increasing attention and emerged as the new paradigm for large language model (LLM) evaluation. Unlike traditional fine-tuning…
In-Context Learning with Many Demonstration Examples
Mukai Li, Shansan Gong, Jiangtao Feng +4
Large pre-training language models (PLMs) have shown promising in-context learning abilities. However, due to the backbone transformer architecture, existing PLMs are bottlenecked…
Compositional Exemplars for In-context Learning
Jiacheng Ye, Zhiyong Wu, Jiangtao Feng +2
Large pretrained language models (LMs) have shown impressive In-Context Learning (ICL) ability, where the model learns to do an unseen task via a prompt consisting of input-output…
A Survey on In-context Learning
Qingxiu Dong, Lei Li, Damai Dai +10
With the increasing capabilities of large language models (LLMs), in-context learning (ICL) has emerged as a new paradigm for natural language processing (NLP), where LLMs make pre…
Self-Adaptive In-Context Learning: An Information Compression Perspective for In-Context Example Selection and Ordering
Zhiyong Wu, Yaoxiang Wang, Jiacheng Ye +1
Despite the surprising few-shot performance of in-context learning (ICL), it is still a common practice to randomly sample examples to serve as context. This paper advocates a new…
Explanation Regeneration via Information Bottleneck
Qintong Li, Zhiyong Wu, Lingpeng Kong +1
Explaining the black-box predictions of NLP models naturally and accurately is an important open problem in natural language generation. These free-text explanations are expected t…
Unsupervised Explanation Generation via Correct Instantiations
Sijie Cheng, Zhiyong Wu, Jiangjie Chen +3
While large pre-trained language models (PLM) have shown their great skills at solving discriminative tasks, a significant gap remains when compared with humans for explanation-rel…
ProGen: Progressive Zero-shot Dataset Generation via In-context Feedback
Jiacheng Ye, Jiahui Gao, Jiangtao Feng +3
Recently, dataset-generation-based zero-shot learning has shown promising results by training a task-specific model with a dataset synthesized from large pre-trained language model…
DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models
Shansan Gong, Mukai Li, Jiangtao Feng +2
Recently, diffusion models have emerged as a new paradigm for generative models. Despite the success in domains using continuous signals such as vision and audio, adapting diffusio…
COLO: A Contrastive Learning based Re-ranking Framework for One-Stage Summarization
Chenxin An, Ming Zhong, Zhiyong Wu +3
Traditional training paradigms for extractive and abstractive summarization systems always only use token-level or sentence-level training objectives. However, the output summary i…
Lexical Knowledge Internalization for Neural Dialog Generation
Zhiyong Wu, Wei Bi, Xiang Li +2
We propose knowledge internalization (KI), which aims to complement the lexical knowledge into neural dialog models. Instead of further conditioning the knowledge-grounded dialog (…
Self-Guided Noise-Free Data Generation for Efficient Zero-Shot Learning
Jiahui Gao, Renjie Pi, Yong Lin +7
There is a rising interest in further exploring the zero-shot learning potential of large pre-trained language models (PLMs). A new paradigm called data-generation-based zero-shot…
ZeroGen: Efficient Zero-shot Learning via Dataset Generation
Jiacheng Ye, Jiahui Gao, Qintong Li +5
There is a growing interest in dataset generation recently due to the superior generative capacity of large pre-trained language models (PLMs). In this paper, we study a flexible a…
Cascaded Head-colliding Attention
Lin Zheng, Zhiyong Wu, Lingpeng Kong
Transformers have advanced the field of natural language processing (NLP) on a variety of important tasks. At the cornerstone of the Transformer architecture is the multi-head atte…
Good for Misconceived Reasons: An Empirical Revisiting on the Need for Visual Context in Multimodal Machine Translation
Zhiyong Wu, Lingpeng Kong, Wei Bi +2
A neural multimodal machine translation (MMT) system is one that aims to perform better translation by extending conventional text-only translation models with multimodal informati…
Perturbed Masking: Parameter-free Probing for Analyzing and Interpreting BERT
Zhiyong Wu, Yun Chen, Ben Kao +1
By introducing a small set of additional parameters, a probe learns to solve specific linguistic tasks (e.g., dependency parsing) in a supervised manner using feature representatio…