58 citations · 180 across the 37 of their papers we have counts for
11 papers · 1 filter
Mark, Don't Erase: Token Inoculation for Dual-Use Knowledge in LLMs
Seunghyun Lee, Dongyoon Han, Sangdoo Yun
Safety interventions on dual-use knowledge typically choose between destroying hazardous content (e.g., unlearning, filtering) and suppressing it at the output layer (e.g., refusal…
On-Policy Delta Distillation
Byeongho Heo, Jaehui Hwang, Sangdoo Yun +1
On-policy distillation is an alternative post-training method in reinforcement learning that alleviates the constraints imposed by reward models by providing token-level supervisio…
Fast KVzip: Efficient and Accurate LLM Inference with Gated KV Eviction
Jang-Hyun Kim, Dongyoon Han, Sangdoo Yun
Efficient key-value (KV) cache management is crucial for the practical deployment of large language models (LLMs), yet existing compression techniques often incur a trade-off betwe…
DNNs May Determine Major Properties of Their Outputs Early, with Timing Possibly Driven by Bias
Song Park, Sanghyuk Chun, Byeongho Heo +1
This paper argues that deep neural networks (DNNs) mostly determine their outputs during the early stages of inference, where biases inherent in the model play a crucial role in sh…
Peri-LN: Revisiting Normalization Layer in the Transformer Architecture
Jeonghoon Kim, Byeongchan Lee, Cheonbok Park +7
Selecting a layer normalization (LN) strategy that stabilizes training and speeds convergence in Transformers remains difficult, even for today's large language models (LLM). We pr…
SyMerge: From Non-Interference to Synergistic Merging via Single-Layer Adaptation
Aecheon Jung, Seunghwan Lee, Dongyoon Han +1
Model merging combines independently trained models into a single multi-task model. However, most existing approaches focus primarily on avoiding task interference. We argue that i…