7 papers
Sentence Curve Language Models
DongNyeong Heo, Taehwan Kim, Heeyoul Choi
Language models (LMs) are a central component of modern AI systems, and diffusion language models (DLMs) have recently emerged as a competitive alternative. Both paradigms rely on…
Dynamic Preference Multi-Objective Reinforcement Learning for Internet Network Management
DongNyeong Heo, Daniela Noemi Rim, Heeyoul Choi
An internet network service provider manages its network with multiple objectives, such as high quality of service (QoS) and minimum computing resource usage. To achieve these obje…
Data Augmentation With Back translation for Low Resource languages: A case of English and Luganda
Richard Kimera, Dongnyeong Heo, Daniela N. Rim +1
In this paper,we explore the application of Back translation (BT) as a semi-supervised technique to enhance Neural Machine Translation(NMT) models for the English-Luganda language…
Generalized Probabilistic Attention Mechanism in Transformers
DongNyeong Heo, Heeyoul Choi
The Transformer architecture has become widely adopted due to its demonstrated success, attributed to the attention mechanism at its core. Despite these successes, the attention me…
Shared Latent Space by Both Languages in Non-Autoregressive Neural Machine Translation
DongNyeong Heo, Heeyoul Choi
Non-autoregressive neural machine translation (NAT) offers substantial translation speed up compared to autoregressive neural machine translation (AT) at the cost of translation qu…
N-gram Prediction and Word Difference Representations for Language Modeling
DongNyeong Heo, Daniela Noemi Rim, Heeyoul Choi
Causal language modeling (CLM) serves as the foundational framework underpinning remarkable successes of recent large language models (LLMs). Despite its success, the training appr…