16 citations · 20 across the 9 of their papers we have counts for
10 papers
WASD: Wasserstein-based Knowledge Distillation for Large Language Models
Byeonghu Na, Donghyeok Shin, Yeongmin Kim +2
Autoregressive large language models (LLMs) have rapidly advanced in capability, but their increasing scale comes with substantial computational and memory costs at inference time.…
Lookahead Sample Reward Guidance for Test-Time Scaling of Diffusion Models
Yeongmin Kim, Donghyeok Shin, Byeonghu Na +3
Diffusion models have demonstrated strong generative performance; however, generated samples often fail to fully align with human intent. This paper studies an efficient test-time…
Diffusion Adaptive Text Embedding for Text-to-Image Diffusion Models
Byeonghu Na, Minsang Park, Gyuwon Sim +6
Text-to-image diffusion models rely on text embeddings from a pre-trained text encoder, but these embeddings remain fixed across all diffusion timesteps, limiting their adaptabilit…
AMiD: Knowledge Distillation for LLMs with -mixture Assistant Distribution
Donghyeok Shin, Yeongmin Kim, Suhyeon Jo +2
Autoregressive large language models (LLMs) have achieved remarkable improvement across many tasks but incur high computational and memory costs. Knowledge distillation (KD) mitiga…
Distillation of Large Language Models via Concrete Score Matching
Yeongmin Kim, Donghyeok Shin, Mina Kang +2
Large language models (LLMs) deliver remarkable performance but are costly to deploy, motivating knowledge distillation (KD) for efficient inference. Existing KD objectives typical…
Distilling Dataset into Neural Field
Donghyeok Shin, HeeSun Bae, Gyuwon Sim +2
Utilizing a large-scale dataset is essential for training high-performance deep learning models, but it also comes with substantial computation and storage costs. To overcome these…
Diffusion Rejection Sampling
Byeonghu Na, Yeongmin Kim, Minsang Park +3
Recent advances in powerful pre-trained diffusion models encourage the development of methods to improve the sampling performance under well-trained diffusion models. This paper in…
Frequency Domain-based Dataset Distillation
Donghyeok Shin, Seungjae Shin, Il-Chul Moon
This paper presents FreD, a novel parameterization method for dataset distillation, which utilizes the frequency domain to distill a small-sized synthetic dataset from a large-size…
Loss-Curvature Matching for Dataset Selection and Condensation
Seungjae Shin, Heesun Bae, Donghyeok Shin +2
Training neural networks on a large dataset requires substantial computational costs. Dataset reduction selects or synthesizes data instances based on the large dataset, while mini…
Unknown-Aware Domain Adversarial Learning for Open-Set Domain Adaptation
JoonHo Jang, Byeonghu Na, DongHyeok Shin +3
Open-Set Domain Adaptation (OSDA) assumes that a target domain contains unknown classes, which are not discovered in a source domain. Existing domain adversarial learning methods a…