94 citations · 325 across the 10 of their papers we have counts for
7 papers · 1 filter
Tgea: An error-annotated dataset and benchmark tasks for text generation from pretrained language models
Jie He, Bo Peng, Yi Liao +2
In order to deeply understand the capability of pretrained language models in text generation and conduct a diagnostic evaluation, we propose TGEA, an error-annotated dataset with…
PARP: Prune, Adjust and Re-Prune for Self-Supervised Speech Recognition
Cheng-I Jeff Lai, Yang Zhang, Alexander H. Liu +7
Self-supervised speech representation learning (speech SSL) has demonstrated the benefit of scale in learning rich representations for Automatic Speech Recognition (ASR) with limit…
PanGu-: Large-scale Autoregressive Pretrained Chinese Language Models with Auto-parallel Computation
Wei Zeng, Xiaozhe Ren, Teng Su +35
Large-scale Pretrained Language Models (PLMs) have become the new paradigm for Natural Language Processing (NLP). PLMs with hundreds of billions parameters such as GPT-3 have demon…
Probabilistically Masked Language Model Capable of Autoregressive Generation in Arbitrary Word Order
Yi Liao, Xin Jiang, Qun Liu
Masked language model and autoregressive language model are two types of language models. While pretrained masked language models such as BERT overwhelm the line of natural languag…
Zero-Shot Paraphrase Generation with Multilingual Language Models
Yinpeng Guo, Yi Liao, Xin Jiang +3
Leveraging multilingual parallel texts to automatically generate paraphrases has drawn much attention as size of high-quality paraphrase corpus is limited. Round-trip translation,…
NEZHA: Neural Contextualized Representation for Chinese Language Understanding
Junqiu Wei, Xiaozhe Ren, Xiaoguang Li +7
The pre-trained language models have achieved great successes in various natural language understanding (NLU) tasks due to its capacity to capture the deep contextualized informati…
GPT-based Generation for Classical Chinese Poetry
Yi Liao, Yasheng Wang, Qun Liu +1
We present a simple yet effective method for generating high quality classical Chinese poetry with Generative Pre-trained Language Model (GPT). The method adopts a simple GPT model…