13 citations · 30 across the 6 of their papers we have counts for
7 papers · 1 filter
A Short Study on Compressing Decoder-Based Language Models
Tianda Li, Yassir El Mesbahi, Ivan Kobyzev +6
Pre-trained Language Models (PLMs) have been successful for a wide range of natural language processing (NLP) tasks. The state-of-the-art of PLMs, however, are extremely large to b…
How to Select One Among All? An Extensive Empirical Study Towards the Robustness of Knowledge Distillation in Natural Language Understanding
Tianda Li, Ahmad Rashid, Aref Jafari +3
Knowledge Distillation (KD) is a model compression algorithm that helps transfer the knowledge of a large neural network into a smaller one. Even though KD has shown promise on a w…
Unsupervised Pre-training with Structured Knowledge for Improving Natural Language Inference
Xiaoyu Yang, Xiaodan Zhu, Zhan Shi +1
While recent research on natural language inference has considerably benefited from large annotated datasets, the amount of inference-related knowledge (including commonsense) prov…
Learning to Retrieve Entity-Aware Knowledge and Generate Responses with Copy Mechanism for Task-Oriented Dialogue Systems
Chao-Hong Tan, Xiaoyu Yang, Zi'ou Zheng +7
Task-oriented conversational modeling with unstructured knowledge access, as track 1 of the 9th Dialogue System Technology Challenges (DSTC 9), requests to build a system to genera…
Pre-Trained and Attention-Based Neural Networks for Building Noetic Task-Oriented Dialogue Systems
Jia-Chen Gu, Tianda Li, Quan Liu +3
The NOESIS II challenge, as the Track 2 of the 8th Dialogue System Technology Challenges (DSTC 8), is the extension of DSTC 7. This track incorporates new elements that are vital f…
Speaker-Aware BERT for Multi-Turn Response Selection in Retrieval-Based Chatbots
Jia-Chen Gu, Tianda Li, Quan Liu +4
In this paper, we study the problem of employing pre-trained language models for multi-turn response selection in retrieval-based chatbots. A new model, named Speaker-Aware BERT (S…