8 citations · 10 across the 6 of their papers we have counts for
6 papers
Stable Language Model Pre-training by Reducing Embedding Variability
Woojin Chung, Jiwoo Hong, Na Min An +2
Stable pre-training is essential for achieving better-performing language models. However, tracking pre-training stability by calculating gradient variance at every step is impract…
ORPO: Monolithic Preference Optimization without Reference Model
Jiwoo Hong, Noah Lee, James Thorne
While recent preference alignment algorithms for language models have demonstrated promising results, supervised fine-tuning (SFT) remains imperative for achieving successful conve…
Re3val: Reinforced and Reranked Generative Retrieval
EuiYul Song, Sangryul Kim, Haeju Lee +2
Generative retrieval models encode pointers to information in a corpus as an index within the model's parameters. These models serve as part of a larger pipeline, where retrieved i…
Detrimental Contexts in Open-Domain Question Answering
Philhoon Oh, James Thorne
For knowledge intensive NLP tasks, it has been widely accepted that accessing more information is a contributing factor to improvements in the model's end-to-end performance. Howev…
Knowledge Corpus Error in Question Answering
Yejoon Lee, Philhoon Oh, James Thorne
Recent works in open-domain question answering (QA) have explored generating context passages from large language models (LLMs), replacing the traditional retrieval step in the QA…
VisAlign: Dataset for Measuring the Degree of Alignment between AI and Humans in Visual Perception
Jiyoung Lee, Seungho Kim, Seunghyun Won +6
AI alignment refers to models acting towards human-intended goals, preferences, or ethical principles. Given that most large-scale deep learning models act as black boxes and canno…