559 citations · 2.2k across the 49 of their papers we have counts for
50 papers
Emage: Non-Autoregressive Text-to-Image Generation
Zhangyin Feng, Runyi Hu, Liangxin Liu +7
Autoregressive and diffusion models drive the recent breakthroughs on text-to-image generation. Despite their huge success of generating high-realistic images, a common shortcoming…
SkillNet-X: A Multilingual Multitask Model with Sparsely Activated Skills
Zhangyin Feng, Yong Dai, Fan Zhang +6
Traditional multitask learning methods basically can only exploit common knowledge in task- or language-wise, which lose either cross-language or cross-task knowledge. This paper p…
Improved Visual Story Generation with Adaptive Context Modeling
Zhangyin Feng, Yuchen Ren, Xinmiao Yu +4
Diffusion models developed on top of powerful text-to-image generation models like Stable Diffusion achieve remarkable success in visual story generation. However, the best-perform…
STOA-VLP: Spatial-Temporal Modeling of Object and Action for Video-Language Pre-training
Weihong Zhong, Mao Zheng, Duyu Tang +4
Although large-scale video-language pre-training models, which usually build a global alignment between the video and the text, have achieved remarkable progress on various downstr…
Effidit: Your AI Writing Assistant
Shuming Shi, Enbo Zhao, Duyu Tang +10
In this technical report, we introduce Effidit (Efficient and Intelligent Editing), a digital writing assistant that facilitates users to write higher-quality text more efficiently…
One Model, Multiple Modalities: A Sparsely Activated Approach for Text, Sound, Image, Video and Code
Yong Dai, Duyu Tang, Liangxin Liu +7
People perceive the world with multiple senses (e.g., through hearing sounds, reading words and seeing objects). However, most existing AI systems only process an individual modali…
SkillNet-NLG: General-Purpose Natural Language Generation with a Sparsely Activated Approach
Junwei Liao, Duyu Tang, Fan Zhang +1
We present SkillNet-NLG, a sparsely activated approach that handles many natural language generation tasks with one model. Different from traditional dense models that always activ…
Pretraining Chinese BERT for Detecting Word Insertion and Deletion Errors
Cong Zhou, Yong Dai, Duyu Tang +4
Chinese BERT models achieve remarkable progress in dealing with grammatical errors of word substitution. However, they fail to handle word insertion and deletion because BERT assum…
"Is Whole Word Masking Always Better for Chinese BERT?": Probing on Chinese Grammatical Error Correction
Yong Dai, Linyang Li, Cong Zhou +5
Whole word masking (WWM), which masks all subwords corresponding to a word at once, makes a better English BERT model. For the Chinese language, however, there is no subword becaus…
Exploring and Adapting Chinese GPT to Pinyin Input Method
Minghuan Tan, Yong Dai, Duyu Tang +5
While GPT has become the de-facto method for text generation tasks, its application to pinyin input method remains unexplored. In this work, we make the first exploration to levera…
MarkBERT: Marking Word Boundaries Improves Chinese BERT
Linyang Li, Yong Dai, Duyu Tang +3
We present a Chinese BERT model dubbed MarkBERT that uses word information in this work. Existing word-based BERT models regard words as basic units, however, due to the vocabulary…
SkillNet-NLU: A Sparsely Activated Model for General-Purpose Natural Language Understanding
Fan Zhang, Duyu Tang, Yong Dai +3
Prevailing deep models are single-purpose and overspecialize at individual tasks. However, when being extended to new tasks, they typically forget previously learned skills and lea…
Pretraining without Wordpieces: Learning Over a Vocabulary of Millions of Words
Zhangyin Feng, Duyu Tang, Cong Zhou +6
The standard BERT adopts subword-based tokenization, which may break a word into two or more wordpieces (e.g., converting "lossless" to "loss" and "less"). This will bring inconven…
CoSQA: 20,000+ Web Queries for Code Search and Question Answering
Junjie Huang, Duyu Tang, Linjun Shou +5
Finding codes given natural language query isb eneficial to the productivity of software developers. Future progress towards better semantic matching between query and code require…
Logic-Driven Context Extension and Data Augmentation for Logical Reasoning of Text
Siyuan Wang, Wanjun Zhong, Duyu Tang +5
Logical reasoning of text requires understanding critical logical information in the text and performing inference over them. Large-scale pre-trained models for logical reasoning m…
AR-LSAT: Investigating Analytical Reasoning of Text
Wanjun Zhong, Siyuan Wang, Duyu Tang +6
Analytical reasoning is an essential and challenging task that requires a system to analyze a scenario involving a set of particular circumstances and perform reasoning over it to…
WhiteningBERT: An Easy Unsupervised Sentence Embedding Approach
Junjie Huang, Duyu Tang, Wanjun Zhong +5
Producing the embedding of a sentence in an unsupervised way is valuable to natural language matching and retrieval problems in practice. In this work, we conduct a thorough examin…
CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation
Shuai Lu, Daya Guo, Shuo Ren +19
Benchmark datasets have a significant impact on accelerating research in programming language tasks. In this paper, we introduce CodeXGLUE, a benchmark dataset to foster machine le…
Syntax-Enhanced Pre-trained Model
Zenan Xu, Daya Guo, Duyu Tang +7
We study the problem of leveraging the syntactic structure of text to enhance pre-trained models such as BERT and RoBERTa. Existing methods utilize syntax of text either in the pre…
Neural Deepfake Detection with Factual Structure of Text
Wanjun Zhong, Duyu Tang, Zenan Xu +5
Deepfake detection, the task of automatically discriminating machine-generated text, is increasingly critical with recent advances in natural language generative models. Existing a…
CodeBLEU: a Method for Automatic Evaluation of Code Synthesis
Shuo Ren, Daya Guo, Shuai Lu +7
Evaluation metrics play a vital role in the growth of an area as it defines the standard of distinguishing between good and bad models. In the area of code synthesis, the commonly…
GraphCodeBERT: Pre-training Code Representations with Data Flow
Daya Guo, Shuo Ren, Shuai Lu +15
Pre-trained models for programming language have achieved dramatic empirical improvements on a variety of code-related tasks such as code search, code completion, code summarizatio…
Evidence-Aware Inferential Text Generation with Vector Quantised Variational AutoEncoder
Daya Guo, Duyu Tang, Nan Duan +3
Generating inferential texts about an event in different perspectives requires reasoning over different contexts that the event occurs. Existing works usually ignore the context th…
LogicalFactChecker: Leveraging Logical Operations for Fact Checking with Graph Module Network
Wanjun Zhong, Duyu Tang, Zhangyin Feng +7
Verifying the correctness of a textual statement requires not only semantic reasoning about the meaning of words, but also symbolic reasoning about logical operations like count, s…