416 citations · 640 across the 48 of their papers we have counts for
6 papers · 1 filter
Model Compression with Two-stage Multi-teacher Knowledge Distillation for Web Question Answering System
Ze Yang, Linjun Shou, Ming Gong +2
Deep pre-training and fine-tuning models (such as BERT and OpenAI GPT) have demonstrated excellent results in question answering areas. However, due to the sheer amount of model pa…
Unicoder: A Universal Language Encoder by Pre-training with Multiple Cross-lingual Tasks
Haoyang Huang, Yaobo Liang, Nan Duan +4
We present Unicoder, a universal language encoder that is insensitive to different languages. Given an arbitrary NLP task, a model can be trained with Unicoder using training data…
Graph-Based Reasoning over Heterogeneous External Knowledge for Commonsense Question Answering
Shangwen Lv, Daya Guo, Jingjing Xu +7
Commonsense question answering aims to answer questions which require background knowledge that is not explicitly expressed in the question. The key challenge is how to obtain evid…
Model Compression with Multi-Task Knowledge Distillation for Web-scale Question Answering System
Ze Yang, Linjun Shou, Ming Gong +2
Deep pre-training and fine-tuning models (like BERT, OpenAI GPT) have demonstrated excellent results in question answering areas. However, due to the sheer amount of model paramete…
NeuronBlocks: Building Your NLP DNN Models Like Playing Lego
Ming Gong, Linjun Shou, Wutao Lin +5
Deep Neural Networks (DNN) have been widely employed in industry to address various Natural Language Processing (NLP) tasks. However, many engineers find it a big overhead when the…
Pretraining-Based Natural Language Generation for Text Summarization
Haoyu Zhang, Jianjun Xu, Ji Wang
In this paper, we propose a novel pretraining-based encoder-decoder framework, which can generate the output sequence based on the input sequence in a two-stage manner. For the enc…