38 citations · 79 across the 23 of their papers we have counts for
23 papers
TaRA: Training-Aware Low-Rank Adaptation Initialization
Taehyeon Kim, Eunhyeok Park
Low-Rank Adaptation (LoRA) has become a de facto standard for parameter-efficient fine-tuning (PEFT), yet its performance is highly sensitive to initialization due to the informati…
Not the Dimension, the Norm: What Matters in Gradient-Free Weight Perturbation of Language Models
Taeyeong Kim, Ahhyun Kim, TaeHyeon Kim +1
Adapting a language model to a task no longer requires training all of its weights, and a line of parameter-efficient methods has driven the trainable count from billions down to a…
MERIT Feedback Elicits Better Bargaining in LLM Negotiators
Jihwan Oh, Murad Aghazada, Yooju Shin +2
Bargaining is often regarded as a logical arena rather than an art or a matter of intuition, yet Large Language Models (LLMs) still struggle to navigate it due to limited strategic…
LLM Agents for Bargaining with Utility-based Feedback
Jihwan Oh
Bargaining, a critical aspect of real-world interactions, presents challenges for large language models (LLMs) due to limitations in strategic depth and adaptation to complex human…
Guiding Reasoning in Small Language Models with LLM Assistance
Yujin Kim, Euiin Yi, Minu Kim +2
The limited reasoning capabilities of small language models (SLMs) cast doubt on their suitability for tasks demanding deep, multi-step logical deduction. This paper introduces a f…
AdaSTaR: Adaptive Data Sampling for Training Self-Taught Reasoners
Woosung Koh, Wonbeen Oh, Jaein Jang +7
Self-Taught Reasoners (STaR), synonymously known as Rejection sampling Fine-Tuning (RFT), is an integral part of the training pipeline of self-improving reasoning Language Models (…
: Scalable Auto-Feedback for LLM-based Chart Generation
Woosung Koh, Jang Han Yoon, MinHyung Lee +7
Generating high-quality charts with Large Language Models (LLMs) presents significant challenges due to limited data and the high cost of scaling through human curation. $\langle \…
Towards Fast Multilingual LLM Inference: Speculative Decoding and Specialized Drafters
Euiin Yi, Taehyeon Kim, Hongseok Jeung +2
Large language models (LLMs) have revolutionized natural language processing and broadened their applicability across diverse commercial applications. However, the deployment of th…
Block Transformer: Global-to-Local Language Modeling for Fast Inference
Namgyu Ho, Sangmin Bae, Taehyeon Kim +6
We introduce the Block Transformer which adopts hierarchical global-to-local modeling to autoregressive transformers to mitigate the inference bottlenecks associated with self-atte…
Revisiting Early-Learning Regularization When Federated Learning Meets Noisy Labels
Taehyeon Kim, Donggyu Kim, Se-Young Yun
In the evolving landscape of federated learning (FL), addressing label noise presents unique challenges due to the decentralized and diverse nature of data collection across client…
Hypernetwork-Driven Model Fusion for Federated Domain Generalization
Marc Bartholet, Taehyeon Kim, Ami Beuret +2
Federated Learning (FL) faces significant challenges with domain shifts in heterogeneous data, degrading performance. Traditional domain generalization aims to learn domain-invaria…
Leveraging Normalization Layer in Adapters With Progressive Learning and Adaptive Distillation for Cross-Domain Few-Shot Learning
Yongjin Yang, Taehyeon Kim, Se-Young Yun
Cross-domain few-shot learning presents a formidable challenge, as models must be trained on base classes and then tested on novel classes from various domains with only a few samp…
Instructive Decoding: Instruction-Tuned Large Language Models are Self-Refiner from Noisy Instructions
Taehyeon Kim, Joonkee Kim, Gihun Lee +1
While instruction-tuned language models have demonstrated impressive zero-shot generalization, these models often struggle to generate accurate responses when faced with instructio…
Region-Conditioned Orthogonal 3D U-Net for Weather4Cast Competition
Taehyeon Kim, Shinhwan Kang, Hyeonjeong Shin +4
The Weather4Cast competition (hosted by NeurIPS 2022) required competitors to predict super-resolution rain movies in various regions of Europe when low-resolution satellite contex…
Revisiting Architecture-aware Knowledge Distillation: Smaller Models and Faster Search
Taehyeon Kim, Heesoo Myeong, Se-Young Yun
Knowledge Distillation (KD) has recently emerged as a popular method for compressing neural networks. In recent studies, generalized distillation methods that find parameters and a…
Benchmark Dataset for Precipitation Forecasting by Post-Processing the Numerical Weather Prediction
Taehyeon Kim, Namgyu Ho, Donggyu Kim +1
Precipitation forecasting is an important scientific challenge that has wide-reaching impacts on society. Historically, this challenge has been tackled using numerical weather pred…
Supernet Training for Federated Image Classification under System Heterogeneity
Taehyeon Kim, Se-Young Yun
Efficient deployment of deep neural networks across many devices and resource constraints, particularly on edge devices, is one of the most challenging problems in the presence of…
A Survey of Supernet Optimization and its Applications: Spatial and Temporal Optimization for Neural Architecture Search
Stephen Cha, Taehyeon Kim, Hayeon Lee +1
This survey focuses on categorizing and evaluating the methods of supernet optimization in the field of Neural Architecture Search (NAS). Supernet optimization involves training a…
Mold into a Graph: Efficient Bayesian Optimization over Mixed-Spaces
Jaeyeon Ahn, Taehyeon Kim, Seyoung Yun
Real-world optimization problems are generally not just black-box problems, but also involve mixed types of inputs in which discrete and continuous variables coexist. Such mixed-sp…
Comparing Kullback-Leibler Divergence and Mean Squared Error Loss in Knowledge Distillation
Taehyeon Kim, Jaehoon Oh, NakYil Kim +2
Knowledge distillation (KD), transferring knowledge from a cumbersome teacher model to a lightweight student model, has been investigated to design efficient neural architectures.…
FINE Samples for Learning with Noisy Labels
Taehyeon Kim, Jongwoo Ko, Sangwook Cho +2
Modern deep neural networks (DNNs) become frail when the datasets contain noisy (incorrect) class labels. Robust techniques in the presence of noisy labels can be categorized into…
Adaptive Local Bayesian Optimization Over Multiple Discrete Variables
Taehyeon Kim, Jaeyeon Ahn, Nakyil Kim +1
In the machine learning algorithms, the choice of the hyperparameter is often an art more than a science, requiring labor-intensive search with expert experience. Therefore, automa…
Accurate and Fast Federated Learning via Combinatorial Multi-Armed Bandits
Taehyeon Kim, Sangmin Bae, Jin-woo Lee +1
Federated learning has emerged as an innovative paradigm of collaborative machine learning. Unlike conventional machine learning, a global model is collaboratively learned while da…