47 citations · 91 across the 16 of their papers we have counts for
7 papers · 2 filters
TCBERT: A Technical Report for Chinese Topic Classification BERT
Ting Han, Kunhao Pan, Xinyu Chen +5
Bidirectional Encoder Representations from Transformers or BERT~\cite{devlin-etal-2019-bert} has been one of the base models for various NLP tasks due to its remarkable performance…
Zero-Shot Learners for Natural Language Understanding via a Unified Multiple Choice Perspective
Ping Yang, Junjie Wang, Ruyi Gan +6
We propose a new paradigm for zero-shot learners that is format agnostic, i.e., it is compatible with any format and applicable to a list of language tasks, such as text classifica…
Solving Math Word Problems via Cooperative Reasoning induced Language Models
Xinyu Zhu, Junjie Wang, Lin Zhang +4
Large-scale pre-trained language models (PLMs) bring new opportunities to challenging problems, especially those that need high-level intelligence, such as the math word problem (M…
Fengshenbang 1.0: Being the Foundation of Chinese Cognitive Intelligence
Jiaxing Zhang, Ruyi Gan, Junjie Wang +22
Nowadays, foundation models become one of fundamental infrastructures in artificial intelligence, paving ways to the general intelligence. However, the reality presents two urgent…
Towards No.1 in CLUE Semantic Matching Challenge: Pre-trained Language Model Erlangshen with Propensity-Corrected Loss
Junjie Wang, Yuxiang Zhang, Ping Yang +1
This report describes a pre-trained language model Erlangshen with propensity-corrected loss, the No.1 in CLUE Semantic Matching Challenge. In the pre-training stage, we construct…
Unified BERT for Few-shot Natural Language Understanding
Junyu Lu, Ping Yang, Ruyi Gan +2
Even as pre-trained language models share a semantic encoder, natural language understanding suffers from a diversity of output schemas. In this paper, we propose UBERT, a unified…
BioBART: Pretraining and Evaluation of A Biomedical Generative Language Model
Hongyi Yuan, Zheng Yuan, Ruyi Gan +3
Pretrained language models have served as important backbones for natural language processing. Recently, in-domain pretraining has been shown to benefit various domain-specific dow…