15 citations · 18 across the 17 of their papers we have counts for
10 papers · 1 filter
Generative Prompt Internalization
Haebin Shin, Lei Ji, Yeyun Gong +3
Prompts used in recent large language model based applications are often fixed and lengthy, leading to significant computational overhead. To address this challenge, we propose Gen…
Latent Action Pretraining from Videos
Seonghyeon Ye, Joel Jang, Byeongguk Jeon +13
We introduce Latent Action Pretraining for general Action models (LAPA), an unsupervised method for pretraining Vision-Language-Action (VLA) models without ground-truth robot actio…
Knowledge Entropy Decay during Language Model Pretraining Hinders New Knowledge Acquisition
Jiyeon Kim, Hyunji Lee, Hyowon Cho +6
In this work, we investigate how a model's tendency to broadly integrate its parametric knowledge evolves throughout pretraining, and how this behavior affects overall performance,…
RouterRetriever: Routing over a Mixture of Expert Embedding Models
Hyunji Lee, Luca Soldaini, Arman Cohan +2
Information retrieval methods often rely on a single embedding model trained on large, general-domain datasets like MSMARCO. While this approach can produce a retriever with reason…
Self-Explore: Enhancing Mathematical Reasoning in Language Models with Fine-grained Rewards
Hyeonbin Hwang, Doyoung Kim, Seungone Kim +2
Training on large amounts of rationales (i.e., CoT Fine-tuning) is effective at improving the reasoning capabilities of large language models (LLMs). However, acquiring human-autho…
Semiparametric Token-Sequence Co-Supervision
Hyunji Lee, Doyoung Kim, Jihoon Jun +4
In this work, we introduce a semiparametric token-sequence co-supervision training method. It trains a language model by simultaneously leveraging supervision from the traditional…
INSTRUCTIR: A Benchmark for Instruction Following of Information Retrieval Models
Hanseok Oh, Hyunji Lee, Seonghyeon Ye +4
Despite the critical need to align search targets with users' intention, retrievers often only prioritize query information without delving into the users' intended search context.…
Aligning Large Language Models by On-Policy Self-Judgment
Sangkyu Lee, Sungdong Kim, Ashkan Yousefpour +3
Existing approaches for aligning large language models with human preferences face a trade-off that requires a separate reward model (RM) for on-policy learning. In this paper, we…
Rethinking the Role of Proxy Rewards in Language Model Alignment
Sungdong Kim, Minjoon Seo
Learning from human feedback via proxy reward modeling has been studied to align Large Language Models (LLMs) with human values. However, achieving reliable training through that p…
LangBridge: Multilingual Reasoning Without Multilingual Supervision
Dongkeun Yoon, Joel Jang, Sungdong Kim +3
We introduce LangBridge, a zero-shot approach to adapt language models for multilingual reasoning tasks without multilingual supervision. LangBridge operates by bridging two models…