15 citations · 18 across the 17 of their papers we have counts for
3 papers · 1 filter
Understanding and Enhancing Mamba-Transformer Hybrids for Memory Recall and Language Modeling
Hyunji Lee, Wenhao Yu, Hongming Zhang +4
Hybrid models that combine state space models (SSMs) with attention mechanisms have shown strong performance by leveraging the efficiency of SSMs and the high recall ability of att…
Instruction Tuning with and without Context: Behavioral Shifts and Downstream Impact
Hyunji Lee, Seunghyun Yoon, Yunjae Won +7
Instruction tuning is a widely used approach to improve the instruction-following ability of large language models (LLMs). Instruction-tuning datasets typically include a mixture o…
Differential Information Distribution: A Bayesian Perspective on Direct Preference Optimization
Yunjae Won, Hyunji Lee, Hyeonbin Hwang +1
Direct Preference Optimization (DPO) has been widely used for aligning language models with human preferences in a supervised manner. However, several key questions remain unresolv…