1.5k citations · 1.7k across the 24 of their papers we have counts for
24 papers
Neuron-based Personality Trait Induction in Large Language Models
Jia Deng, Tianyi Tang, Yanbin Yin +3
Large language models (LLMs) have become increasingly proficient at simulating various personality traits, an important capability for supporting related applications (e.g., role-p…
LLMBox: A Comprehensive Library for Large Language Models
Tianyi Tang, Yiwen Hu, Bingqian Li +22
To facilitate the research on large language models (LLMs), this paper presents a comprehensive and unified library, LLMBox, to ease the development, use, and evaluation of LLMs. T…
Towards Coarse-to-Fine Evaluation of Inference Efficiency for Large Language Models
Yushuo Chen, Tianyi Tang, Erge Xiang +5
In real world, large language models (LLMs) can serve as the assistant to help users accomplish their jobs, and also support the development of advanced applications. For the wide…
Language-Specific Neurons: The Key to Multilingual Capabilities in Large Language Models
Tianyi Tang, Wenyang Luo, Haoyang Huang +5
Large language models (LLMs) demonstrate remarkable multilingual capabilities without being pre-trained on specially curated multilingual parallel corpora. It remains a challenging…
Beyond Imitation: Leveraging Fine-grained Quality Signals for Alignment
Geyang Guo, Ranchi Zhao, Tianyi Tang +2
Alignment with human preference is a desired property of large language models (LLMs). Currently, the main alignment approach is based on reinforcement learning from human feedback…
BAMBOO: A Comprehensive Benchmark for Evaluating Long Text Modeling Capacities of Large Language Models
Zican Dong, Tianyi Tang, Junyi Li +2
Large language models (LLMs) have achieved dramatic proficiency over NLP tasks with normal length. Recently, multiple studies have committed to extending the context length and enh…
Towards Effective Ancient Chinese Translation: Dataset, Model, and Evaluation
Geyang Guo, Jiarong Yang, Fengyuan Lu +3
Interpreting ancient Chinese has been the key to comprehending vast Chinese literature, tradition, and civilization. In this paper, we propose Erya for ancient Chinese translation.…
Learning to Imagine: Visually-Augmented Natural Language Generation
Tianyi Tang, Yushuo Chen, Yifan Du +3
People often imagine relevant scenes to aid in the writing process. In this work, we aim to utilize visual information for composition in the same manner as humans. We propose a me…
Zero-shot Visual Question Answering with Language Model Feedback
Yifan Du, Junyi Li, Tianyi Tang +2
In this paper, we propose a novel language model guided captioning approach, LAMOC, for knowledge-based visual question answering (VQA). Our approach employs the generated captions…
The Web Can Be Your Oyster for Improving Large Language Models
Junyi Li, Tianyi Tang, Wayne Xin Zhao +3
Large language models (LLMs) encode a large amount of world knowledge. However, as such knowledge is frozen at the time of model training, the models become static and limited by t…
Not All Metrics Are Guilty: Improving NLG Evaluation by Diversifying References
Tianyi Tang, Hongyuan Lu, Yuchen Eleanor Jiang +5
Most research about natural language generation (NLG) relies on evaluation benchmarks with limited references for a sample, which may result in poor correlations with human judgeme…
Not All Languages Are Created Equal in LLMs: Improving Multilingual Capability by Cross-Lingual-Thought Prompting
Haoyang Huang, Tianyi Tang, Dongdong Zhang +4
Large language models (LLMs) demonstrate impressive multilingual capability, but their performance varies substantially across different languages. In this work, we introduce a sim…
A Survey of Large Language Models
Wayne Xin Zhao, Kun Zhou, Junyi Li +19
Language is essentially a complex, intricate system of human expressions governed by grammatical rules. It poses a significant challenge to develop capable AI algorithms for compre…
A Survey on Long Text Modeling with Transformers
Zican Dong, Tianyi Tang, Junyi Li +1
Modeling long texts has been an essential technique in the field of natural language processing (NLP). With the ever-growing number of long documents, it is important to develop ef…
TextBox 2.0: A Text Generation Library with Pre-trained Language Models
Tianyi Tang, Junyi Li, Zhipeng Chen +9
To facilitate research on text generation, this paper presents a comprehensive and unified library, TextBox 2.0, focusing on the use of pre-trained language models (PLMs). To be co…
ELMER: A Non-Autoregressive Pre-trained Language Model for Efficient and Effective Text Generation
Junyi Li, Tianyi Tang, Wayne Xin Zhao +2
We study the text generation task under the approach of pre-trained language models (PLMs). Typically, an auto-regressive (AR) method is adopted for generating texts in a token-by-…
MVP: Multi-task Supervised Pre-training for Natural Language Generation
Tianyi Tang, Junyi Li, Wayne Xin Zhao +1
Pre-trained language models (PLMs) have achieved remarkable success in natural language generation (NLG) tasks. Up to now, most NLG-oriented PLMs are pre-trained in an unsupervised…
Learning to Transfer Prompts for Text Generation
Junyi Li, Tianyi Tang, Jian-Yun Nie +2
Pretrained language models (PLMs) have made remarkable progress in text generation tasks via fine-tuning. While, it is challenging to fine-tune PLMs in a data-scarce situation. The…
ElitePLM: An Empirical Study on General Language Ability Evaluation of Pretrained Language Models
Junyi Li, Tianyi Tang, Zheng Gong +6
Nowadays, pretrained language models (PLMs) have dominated the majority of NLP tasks. While, little research has been conducted on systematically evaluating the language abilities…
Context-Tuning: Learning Contextualized Prompts for Natural Language Generation
Tianyi Tang, Junyi Li, Wayne Xin Zhao +1
Recently, pretrained language models (PLMs) have had exceptional success in language generation. To leverage the rich knowledge encoded by PLMs, a simple yet powerful paradigm is t…
Pretrained Language Models for Text Generation: A Survey
Junyi Li, Tianyi Tang, Wayne Xin Zhao +2
Text Generation aims to produce plausible and readable text in a human language from input data. The resurgence of deep learning has greatly advanced this field, in particular, wit…
Few-shot Knowledge Graph-to-Text Generation with Pretrained Language Models
Junyi Li, Tianyi Tang, Wayne Xin Zhao +3
This paper studies how to automatically generate a natural language text that describes the facts in knowledge graph (KG). Considering the few-shot setting, we leverage the excelle…
Pretrained Language Models for Text Generation: A Survey
Junyi Li, Tianyi Tang, Wayne Xin Zhao +1
Text generation has become one of the most important yet challenging tasks in natural language processing (NLP). The resurgence of deep learning has greatly advanced this field by…
TextBox: A Unified, Modularized, and Extensible Framework for Text Generation
Junyi Li, Tianyi Tang, Gaole He +7
In this paper, we release an open-source library, called TextBox, to provide a unified, modularized, and extensible text generation framework. TextBox aims to support a broad set o…