193 citations · 1.5k across the 86 of their papers we have counts for
14 papers · 1 filter
LiDAM: Semi-Supervised Learning with Localized Domain Adaptation and Iterative Matching
Qun Liu, Matthew Shreve, Raja Bala
Although data is abundant, data labeling is expensive. Semi-supervised learning methods combine a few labeled samples with a large corpus of unlabeled data to effectively train mod…
From Unsupervised Machine Translation To Adversarial Text Generation
Ahmad Rashid, Alan Do-Omri, Md. Akmal Haidar +2
We present a self-attention based bilingual adversarial text generator (B-GAN) which can learn to generate text from the encoder representation of an unsupervised neural machine tr…
Know What You Don't Need: Single-Shot Meta-Pruning for Attention Heads
Zhengyan Zhang, Fanchao Qi, Zhiyuan Liu +2
Deep pre-trained Transformer models have achieved state-of-the-art results over a variety of natural language processing (NLP) tasks. By learning rich language knowledge with milli…
TernaryBERT: Distillation-aware Ultra-low Bit BERT
Wei Zhang, Lu Hou, Yichun Yin +4
Transformer-based pre-training models like BERT have achieved remarkable performance in many natural language processing tasks.However, these models are both computation and memory…
Why Skip If You Can Combine: A Simple Knowledge Distillation Technique for Intermediate Layers
Yimeng Wu, Peyman Passban, Mehdi Rezagholizade +1
With the growth of computing power neural machine translation (NMT) models also grow accordingly and become better. However, they also become harder to deploy on edge devices due t…
SparTerm: Learning Term-based Sparse Representation for Fast Text Retrieval
Yang Bai, Xiaoguang Li, Gang Wang +6
Term-based sparse representations dominate the first-stage text retrieval in industrial applications, due to its advantage in efficiency, interpretability, and exact term matching.…
HyperText: Endowing FastText with Hyperbolic Geometry
Yudong Zhu, Di Zhou, Jinghui Xiao +3
Natural language data exhibit tree-like hierarchical structures such as the hypernym-hyponym relations in WordNet. FastText, as the state-of-the-art text classifier based on shallo…
TensorCoder: Dimension-Wise Attention via Tensor Representation for Natural Language Modeling
Shuai Zhang, Peng Zhang, Xindian Ma +3
Transformer has been widely-used in many Natural Language Processing (NLP) tasks and the scaled dot-product attention between tokens is a core module of Transformer. This attention…
Learning to Detect Unacceptable Machine Translations for Downstream Tasks
Meng Zhang, Xin Jiang, Yang Liu +1
The field of machine translation has progressed tremendously in recent years. Even though the translation quality has improved significantly, current systems are still unable to pr…
Probabilistically Masked Language Model Capable of Autoregressive Generation in Arbitrary Word Order
Yi Liao, Xin Jiang, Qun Liu
Masked language model and autoregressive language model are two types of language models. While pretrained masked language models such as BERT overwhelm the line of natural languag…
Dictionary-based Data Augmentation for Cross-Domain Neural Machine Translation
Wei Peng, Chongxuan Huang, Tianhao Li +2
Existing data augmentation approaches for neural machine translation (NMT) have predominantly relied on back-translating in-domain (IND) monolingual corpora. These methods suffer f…
Accurate Word Alignment Induction from Neural Machine Translation
Yun Chen, Yang Liu, Guanhua Chen +2
Despite its original goal to jointly learn to align and translate, prior researches suggest that Transformer captures poor word alignments through its attention mechanism. In this…
Perturbed Masking: Parameter-free Probing for Analyzing and Interpreting BERT
Zhiyong Wu, Yun Chen, Ben Kao +1
By introducing a small set of additional parameters, a probe learns to solve specific linguistic tasks (e.g., dependency parsing) in a supervised manner using feature representatio…
DynaBERT: Dynamic BERT with Adaptive Width and Depth
Lu Hou, Zhiqi Huang, Lifeng Shang +3
The pre-trained language models like BERT, though powerful in many natural language processing tasks, are both computation and memory expensive. To alleviate this problem, one appr…