19 citations · 23 across the 5 of their papers we have counts for
Showing 2022Show all
2 papers · 1 filter
cs.CL2022
SkillNet-NLG: General-Purpose Natural Language Generation with a Sparsely Activated Approach
Junwei Liao, Duyu Tang, Fan Zhang +1
We present SkillNet-NLG, a sparsely activated approach that handles many natural language generation tasks with one model. Different from traditional dense models that always activ…
cs.CL2022★ 4 cited
Pretraining without Wordpieces: Learning Over a Vocabulary of Millions of Words
Zhangyin Feng, Duyu Tang, Cong Zhou +6
The standard BERT adopts subword-based tokenization, which may break a word into two or more wordpieces (e.g., converting "lossless" to "loss" and "less"). This will bring inconven…