45 citations · 307 across the 59 of their papers we have counts for
16 papers · 1 filter
LMTurk: Few-Shot Learners as Crowdsourcing Workers in a Language-Model-as-a-Service Framework
Mengjie Zhao, Fei Mi, Yasheng Wang +4
Vast efforts have been devoted to creating high-performance few-shot learners, i.e., large-scale pretrained language models (PLMs) that perform well with little downstream task tra…
CoCA-MDD: A Coupled Cross-Attention based Framework for Streaming Mispronunciation Detection and Diagnosis
Nianzu Zheng, Liqun Deng, Wenyong Huang +7
Mispronunciation detection and diagnosis (MDD) is a popular research focus in computer-aided pronunciation training (CAPT) systems. End-to-end (e2e) approaches are becoming dominan…
UniDS: A Unified Dialogue System for Chit-Chat and Task-oriented Dialogues
Xinyan Zhao, Bin He, Yasheng Wang +6
With the advances in deep learning, tremendous progress has been made with chit-chat dialogue systems and task-oriented dialogue systems. However, these two systems are often tackl…
bert2BERT: Towards Reusable Pretrained Language Models
Cheng Chen, Yichun Yin, Lifeng Shang +7
In recent years, researchers tend to pre-train ever-larger language models to explore the upper limit of deep models. However, large language model pre-training costs intensive com…
NumGPT: Improving Numeracy Ability of Generative Pre-trained Models
Zhihua Jin, Xin Jiang, Xingbo Wang +4
Existing generative pre-trained language models (e.g., GPT) focus on modeling the language structure and semantics of general texts. However, those models do not consider the numer…
Improving Unsupervised Question Answering via Summarization-Informed Question Generation
Chenyang Lyu, Lifeng Shang, Yvette Graham +3
Question Generation (QG) is the task of generating a plausible question for a given <passage, answer> pair. Template-based QG uses linguistically-informed heuristics to transform d…
Generate & Rank: A Multi-task Framework for Math Word Problems
Jianhao Shen, Yichun Yin, Lin Li +4
Math word problem (MWP) is a challenging and critical task in natural language processing. Many recent studies formalize MWP as a generation task and have adopted sequence-to-seque…
Integrating Regular Expressions with Neural Networks via DFA
Shaobo Li, Qun Liu, Xin Jiang +5
Human-designed rules are widely used to build industry applications. However, it is infeasible to maintain thousands of such hand-crafted rules. So it is very important to integrat…
DyLex: Incorporating Dynamic Lexicons into BERT for Sequence Labeling
Baojun Wang, Zhao Zhang, Kun Xu +7
Incorporating lexical knowledge into deep learning models has been proved to be very effective for sequence labeling tasks. However, previous works commonly have difficulty dealing…
UniMS: A Unified Framework for Multimodal Summarization with Knowledge Distillation
Zhengkun Zhang, Xiaojun Meng, Yasheng Wang +3
With the rapid increase of multimedia data, a large body of literature has emerged to work on multimodal summarization, the majority of which target at refining salient information…
CINS: Comprehensive Instruction for Few-shot Learning in Task-oriented Dialog Systems
Fei Mi, Yitong Li, Yasheng Wang +2
As labeling cost for different modules in task-oriented dialog (ToD) systems is high, a major challenge in practice is to learn different tasks with the least amount of labeled dat…
AutoTinyBERT: Automatic Hyper-parameter Optimization for Efficient Pre-trained Language Models
Yichun Yin, Cheng Chen, Lifeng Shang +3
Pre-trained language models (PLMs) have achieved great success in natural language processing. Most of PLMs follow the default setting of architecture hyper-parameters (e.g., the h…
A Mutual Information Maximization Approach for the Spurious Solution Problem in Weakly Supervised Question Answering
Zhihong Shao, Lifeng Shang, Qun Liu +1
Weakly supervised question answering usually has only the final answers as supervision signals while the correct solutions to derive the answers are not provided. This setting give…
Sub-Character Tokenization for Chinese Pretrained Language Models
Chenglei Si, Zhengyan Zhang, Yingfa Chen +6
Tokenization is fundamental to pretrained language models (PLMs). Existing tokenization methods for Chinese PLMs typically treat each character as an indivisible token. However, th…
LightMBERT: A Simple Yet Effective Method for Multilingual BERT Distillation
Xiaoqi Jiao, Yichun Yin, Lifeng Shang +5
The multilingual pre-trained language models (e.g, mBERT, XLM and XLM-R) have shown impressive performance on cross-lingual natural language understanding tasks. However, these mod…
Training Multilingual Pre-trained Language Model with Byte-level Subwords
Junqiu Wei, Qun Liu, Yinpeng Guo +1
The pre-trained language models have achieved great successes in various natural language understanding (NLU) tasks due to its capacity to capture the deep contextualized informati…