1 citations · 1 across the 10 of their papers we have counts for
6 papers · 1 filter
No Pain, More Gain: Iterative Merging for Effective Multi-Teacher On-Policy Distillation
Seonghyeon Kim, Chaeyun Jang, Noah Lee +2
Multi-teacher on-policy distillation (MOPD) combines independently developed domain teachers into a single student by distilling their predictions on student-generated samples. We…
The Distributional View of Knowledge Distillation
Gordei Verbii, Juho Lee
Token-level knowledge distillation (KD) matches two conditional distributions per position, yet the standard objectives compare them pointwise: a Kullback-Leibler gradient is blind…
From Drift to Coherence: Stabilizing Beliefs in LLMs
SongEun Kim, Seungyoo Lee, Edwin Fong +2
Large language models (LLMs) are often hypothesized to perform implicit Bayesian inference, yet a key coherence condition, the martingale property of predictive beliefs, has been s…
Parallel Test-Time Scaling with Multi-Sequence Verifiers
Yegon Kim, Seungyoo Lee, Chaeyun Jang +2
Parallel test-time scaling, which generates multiple candidate solutions for a single problem, is a powerful technique for improving large language model performance. However, it i…
Mitigating Legibility Tax with Decoupled Prover-Verifier Games
Yegon Kim, Juho Lee
As large language models become increasingly capable, it is critical that their outputs can be easily checked by less capable systems. Prover-verifier games can be used to improve…
A Model-Free Universal AI
Yegon Kim, Juho Lee
In general reinforcement learning, all established optimal agents, including AIXI, are model-based, explicitly maintaining and using environment models. This paper introduces Unive…