51 citations · 205 across the 17 of their papers we have counts for
10 papers · 1 filter
CILDA: Contrastive Data Augmentation using Intermediate Layer Knowledge Distillation
Md Akmal Haidar, Mehdi Rezagholizadeh, Abbas Ghaddar +3
Knowledge distillation (KD) is an efficient framework for compressing large-scale pre-trained language models. Recent years have seen a surge of research aiming to improve KD by le…
RAIL-KD: RAndom Intermediate Layer Mapping for Knowledge Distillation
Md Akmal Haidar, Nithin Anchuri, Mehdi Rezagholizadeh +3
Intermediate layer knowledge distillation (KD) can improve the standard KD technique (which only targets the output of teacher and student models) especially over large pre-trained…
Robust Embeddings Via Distributions
Kira A. Selby, Yinong Wang, Ruizhe Wang +4
Despite recent monumental advances in the field, many Natural Language Processing (NLP) models still struggle to perform adequately on noisy domains. We propose a novel probabilist…
Generating Emotionally Aligned Responses in Dialogues using Affect Control Theory
Nabiha Asghar, Ivan Kobyzev, Jesse Hoey +2
State-of-the-art neural dialogue systems excel at syntactic and semantic modelling of language, but often have a hard time establishing emotional alignment with the human interacta…
Unsupervised Multilingual Alignment using Wasserstein Barycenter
Xin Lian, Kshitij Jain, Jakub Truszkowski +2
We study unsupervised multilingual alignment, the problem of finding word-to-word translations between multiple languages without using any parallel data. One popular strategy is t…
Progressive Memory Banks for Incremental Domain Adaptation
Nabiha Asghar, Lili Mou, Kira A. Selby +3
This paper addresses the problem of incremental domain adaptation (IDA) in natural language processing (NLP). We assume each domain comes one after another, and that we could only…