112 citations · 174 across the 13 of their papers we have counts for
6 papers · 1 filter
Adversarial Training with Contrastive Learning in NLP
Daniela N. Rim, DongNyeong Heo, Heeyoul Choi
For years, adversarial training has been extensively studied in natural language processing (NLP) settings. The main goal is to make models robust so that similar inputs derive in…
Self-Knowledge Distillation in Natural Language Processing
Sangchul Hahn, Heeyoul Choi
Since deep learning became a key player in natural language processing (NLP), many deep learning models have been showing remarkable performances in a variety of NLP tasks, and in…
Persistent Hidden States and Nonlinear Transformation for Long Short-Term Memory
Heeyoul Choi
Recurrent neural networks (RNNs) have been drawing much attention with great success in many applications like speech recognition and neural machine translation. Long short-term me…
Fine-Grained Attention Mechanism for Neural Machine Translation
Heeyoul Choi, Kyunghyun Cho, Yoshua Bengio
Neural machine translation (NMT) has been a new paradigm in machine translation, and the attention mechanism has become the dominant approach with the state-of-the-art records in m…
A Neural Knowledge Language Model
Sungjin Ahn, Heeyoul Choi, Tanel Pärnamaa +1
Current language models have a significant limitation in the ability to encode and decode factual knowledge. This is mainly because they acquire such knowledge from statistical co-…
Context-Dependent Word Representation for Neural Machine Translation
Heeyoul Choi, Kyunghyun Cho, Yoshua Bengio
We first observe a potential weakness of continuous vector representations of symbols in neural machine translation. That is, the continuous vector representation, or a word embedd…