206 citations · 624 across the 70 of their papers we have counts for
70 papers
Prompt-Based Length Controlled Generation with Reinforcement Learning
Renlong Jie, Xiaojun Meng, Lifeng Shang +2
Large language models (LLMs) like ChatGPT and GPT-4 have attracted great attention given their surprising performance on a wide range of NLP tasks. Length controlled generation of…
AutoConv: Automatically Generating Information-seeking Conversations with Large Language Models
Siheng Li, Cheng Yang, Yichun Yin +6
Information-seeking conversation, which aims to help users gather information through conversation, has achieved great progress in recent years. However, the research is still stym…
NewsDialogues: Towards Proactive News Grounded Conversation
Siheng Li, Yichun Yin, Cheng Yang +7
Hot news is one of the most popular topics in daily conversations. However, news grounded conversation has long been stymied by the lack of well-designed task definition and scarce…
Learning Summary-Worthy Visual Representation for Abstractive Summarization in Video
Zenan Xu, Xiaojun Meng, Yasheng Wang +4
Multimodal abstractive summarization for videos (MAS) requires generating a concise textual summary to describe the highlights of a video according to multimodal resources, in our…
Enhancing Coherence of Extractive Summarization with Multitask Learning
Renlong Jie, Xiaojun Meng, Lifeng Shang +2
This study proposes a multitask learning architecture for extractive summarization with coherence boosting. The architecture contains an extractive summarizer and coherent discrimi…
PanGu-Σ: Towards Trillion Parameter Language Model with Sparse Heterogeneous Computing
Xiaozhe Ren, Pingyi Zhou, Xinfan Meng +14
The scaling of large language models has greatly improved natural language understanding, generation, and reasoning. In this work, we develop a system that trained a trillion-param…
Wukong-Reader: Multi-modal Pre-training for Fine-grained Visual Document Understanding
Haoli Bai, Zhiguang Liu, Xiaojun Meng +9
Unsupervised pre-training on millions of digital-born or scanned documents has shown promising advances in visual document understanding~(VDU). While various vision-language pre-tr…
KPT: Keyword-guided Pre-training for Grounded Dialog Generation
Qi Zhu, Fei Mi, Zheng Zhang +6
Incorporating external knowledge into the response generation process is essential to building more helpful and reliable dialog agents. However, collecting knowledge-grounded conve…
Retrieval-based Disentangled Representation Learning with Natural Language Supervision
Jiawei Zhou, Xiaoguang Li, Lifeng Shang +3
Disentangled representation learning remains challenging as the underlying factors of variation in the data do not naturally exist. The inherent complexity of real-world data makes…
G-MAP: General Memory-Augmented Pre-trained Language Model for Domain Tasks
Zhongwei Wan, Yichun Yin, Wei Zhang +5
Recently, domain-specific PLMs have been proposed to boost the task performance of specific domains (e.g., biomedical and computer science) by continuing to pre-train general PLMs…
Lexicon-injected Semantic Parsing for Task-Oriented Dialog
Xiaojun Meng, Wenlin Dai, Yasheng Wang +4
Recently, semantic parsing using hierarchical representations for dialog systems has captured substantial attention. Task-Oriented Parse (TOP), a tree representation with intents a…
LiteVL: Efficient Video-Language Learning with Enhanced Spatial-Temporal Modeling
Dongsheng Chen, Chaofan Tao, Lu Hou +3
Recent large-scale video-language pre-trained models have shown appealing performance on various downstream tasks. However, the pre-training process is computationally expensive du…
Pre-training Language Models with Deterministic Factual Knowledge
Shaobo Li, Xiaoguang Li, Lifeng Shang +5
Previous works show that Pre-trained Language Models (PLMs) can capture factual knowledge. However, some analyses reveal that PLMs fail to perform it robustly, e.g., being sensitiv…
PanGu-Coder: Program Synthesis with Function-Level Language Modeling
Fenia Christopoulou, Gerasimos Lampouras, Milan Gritta +19
We present PanGu-Coder, a pretrained decoder-only language model adopting the PanGu-Alpha architecture for text-to-code generation, i.e. the synthesis of programming language solut…
FreeTransfer-X: Safe and Label-Free Cross-Lingual Transfer from Off-the-Shelf Models
Yinpeng Guo, Liangyou Li, Xin Jiang +1
Cross-lingual transfer (CLT) is of various applications. However, labeled cross-lingual corpus is expensive or even inaccessible, especially in the fields where labels are private,…
PERT: A New Solution to Pinyin to Character Conversion Task
Jinghui Xiao, Qun Liu, Xin Jiang +3
Pinyin to Character conversion (P2C) task is the key task of Input Method Engine (IME) in commercial input software for Asian languages, such as Chinese, Japanese, Thai language an…
Revisiting Pre-trained Language Models and their Evaluation for Arabic Natural Language Understanding
Abbas Ghaddar, Yimeng Wu, Sunyam Bagga +11
There is a growing body of work in recent years to develop pre-trained language models (PLMs) for the Arabic language. This work concerns addressing two major problems in existing…
Exploring Extreme Parameter Compression for Pre-trained Language Models
Yuxin Ren, Benyou Wang, Lifeng Shang +2
Recent work explored the potential of large-scale Transformer-based pre-trained models, especially Pre-trained Language Models (PLMs) in natural language processing. This raises ma…
UTC: A Unified Transformer with Inter-Task Contrastive Learning for Visual Dialog
Cheng Chen, Yudong Zhu, Zhenshan Tan +4
Visual Dialog aims to answer multi-round, interactive questions based on the dialog history and image content. Existing methods either consider answer ranking and generating indivi…
Hyperlink-induced Pre-training for Passage Retrieval in Open-domain Question Answering
Jiawei Zhou, Xiaoguang Li, Lifeng Shang +10
To alleviate the data scarcity problem in training question answering systems, recent works propose additional intermediate pre-training for dense passage retrieval (DPR). However,…
CorrectSpeech: A Fully Automated System for Speech Correction and Accent Reduction
Daxin Tan, Liqun Deng, Nianzu Zheng +4
This study propose a fully automated system for speech correction and accent reduction. Consider the application scenario that a recorded speech audio contains certain errors, e.g.…
How Pre-trained Language Models Capture Factual Knowledge? A Causal-Inspired Analysis
Shaobo Li, Xiaoguang Li, Lifeng Shang +6
Recently, there has been a trend to investigate the factual knowledge captured by Pre-trained Language Models (PLMs). Many works show the PLMs' ability to fill in the missing factu…
Enabling Multimodal Generation on CLIP via Vision-Language Knowledge Distillation
Wenliang Dai, Lu Hou, Lifeng Shang +3
The recent large-scale vision-language pre-training (VLP) of dual-stream architectures (e.g., CLIP) with a tremendous amount of image-text pair data, has shown its superiority on v…
Compilable Neural Code Generation with Compiler Feedback
Xin Wang, Yasheng Wang, Yao Wan +7
Automatically generating compilable programs with (or without) natural language descriptions has always been a touchstone problem for computational linguistics and automated softwa…