2 citations · 2 across the 6 of their papers we have counts for
5 papers · 1 filter
Pruning and Distilling Mixture-of-Experts into Dense Language Models
Junhyuck Kim, Jihun Yun, Haechan Kim +3
Mixture-of-Experts (MoE) is now the dominant architecture for frontier language models, yet it requires all expert parameters to be loaded in memory, making it less preferable for…
Distilling LLM Agent into Small Models with Retrieval and Code Tools
Minki Kang, Jongwon Jeong, Seanie Lee +2
Large language models (LLMs) excel at complex reasoning tasks but remain computationally expensive, limiting their practical deployment. To address this, recent works have focused…
T1: Tool-integrated Verification for Test-time Compute Scaling in Small Language Models
Minki Kang, Jongwon Jeong, Jaewoong Cho
Recent studies have demonstrated that test-time compute scaling effectively improves the performance of small language models (sLMs). However, prior research has mainly examined te…
Latent Paraphrasing: Perturbation on Layers Improves Knowledge Injection in Language Models
Minki Kang, Sung Ju Hwang, Gibbeum Lee +1
As Large Language Models (LLMs) are increasingly deployed in specialized domains with continuously evolving knowledge, the need for timely and precise knowledge injection has becom…
Accelerating Multilingual Language Model for Excessively Tokenized Languages
Jimin Hong, Gibbeum Lee, Jaewoong Cho
Recent advancements in large language models (LLMs) have remarkably enhanced performances on a variety of tasks in multiple languages. However, tokenizers in LLMs trained primarily…