1 citations · 1 across the 5 of their papers we have counts for
5 papers
Bounded Hyperbolic Tangent: A Stable and Efficient Alternative to Pre-Layer Normalization in Large Language Models
Hoyoon Byun, Youngjun Choi, Taero Kim +2
Pre-Layer Normalization (Pre-LN) is the de facto choice for large language models (LLMs) and is crucial for stable pretraining and effective transfer learning. However, Pre-LN incu…
MIDUS: Memory-Infused Depth Up-Scaling
Taero Kim, Hoyoon Byun, Youngjun Choi +2
Expanding pre-trained language models offers a practical way to increase capacity without training larger models from scratch. Depth Up-Scaling (DUS) does so by duplicating Transfo…
Adaptive Task Vectors for Large Language Models
Joonseong Kang, Soojeong Lee, Subeen Park +5
In-Context Learning (ICL) enables Large Language Models (LLMs) to perform tasks without parameter updates by conditioning on a few demonstrations provided in the prompt. Despite it…
Sufficient Invariant Learning for Distribution Shift
Taero Kim, Subeen Park, Sungjun Lim +3
Learning robust models under distribution shifts between training and test datasets is a fundamental challenge in machine learning. While learning invariant features across environ…
Learning Fair Representation via Distributional Contrastive Disentanglement
Changdae Oh, Heeji Won, Junhyuk So +4
Learning fair representation is crucial for achieving fairness or debiasing sensitive information. Most existing works rely on adversarial representation learning to inject some in…