38 citations · 79 across the 23 of their papers we have counts for
6 papers · 1 filter
TaRA: Training-Aware Low-Rank Adaptation Initialization
Taehyeon Kim, Eunhyeok Park
Low-Rank Adaptation (LoRA) has become a de facto standard for parameter-efficient fine-tuning (PEFT), yet its performance is highly sensitive to initialization due to the informati…
Not the Dimension, the Norm: What Matters in Gradient-Free Weight Perturbation of Language Models
Taeyeong Kim, Ahhyun Kim, TaeHyeon Kim +1
Adapting a language model to a task no longer requires training all of its weights, and a line of parameter-efficient methods has driven the trainable count from billions down to a…
Guiding Reasoning in Small Language Models with LLM Assistance
Yujin Kim, Euiin Yi, Minu Kim +2
The limited reasoning capabilities of small language models (SLMs) cast doubt on their suitability for tasks demanding deep, multi-step logical deduction. This paper introduces a f…
Towards Fast Multilingual LLM Inference: Speculative Decoding and Specialized Drafters
Euiin Yi, Taehyeon Kim, Hongseok Jeung +2
Large language models (LLMs) have revolutionized natural language processing and broadened their applicability across diverse commercial applications. However, the deployment of th…
Block Transformer: Global-to-Local Language Modeling for Fast Inference
Namgyu Ho, Sangmin Bae, Taehyeon Kim +6
We introduce the Block Transformer which adopts hierarchical global-to-local modeling to autoregressive transformers to mitigate the inference bottlenecks associated with self-atte…
Instructive Decoding: Instruction-Tuned Large Language Models are Self-Refiner from Noisy Instructions
Taehyeon Kim, Joonkee Kim, Gihun Lee +1
While instruction-tuned language models have demonstrated impressive zero-shot generalization, these models often struggle to generate accurate responses when faced with instructio…