4 citations · 10 across the 18 of their papers we have counts for
19 papers
Pivot-SD: Efficient Self-Distillation for Masked Diffusion Language Models
Seo Hyun Kim, Sunwoo Hong, Younwoo Choi +3
Masked diffusion language models (dLMs) offer a promising parallel alternative to autoregressive models for complex reasoning. However, they face a distinct credit-assignment chall…
Eliciting Weak-to-Strong Generalization with On-Policy Reverse Distillation
Youngrok Park, Sangmin Bae, Hojung Jung +6
Weak-to-strong generalization asks whether stronger models can learn from weaker supervisors and surpass them. This is particularly important for successive model generations and m…
PLC-DPO: Posterior Label Correction in Noisy and Ambiguous Preference Optimization
Boryeong Cho, Sumyeong Ahn, Se-Young Yun
Direct Preference Optimization (DPO) simplifies alignment through pairwise comparisons but assumes all observed preferences are reliable. Real data often violates this assumption,…
Bastion: Budget-Aware Speculative Decoding with Tree-structured Block Diffusion Drafting
Soowon Oh, Nam Cao, Yujin Kim +4
Block-diffusion drafters have recently emerged as a powerful alternative for speculative decoding by predicting multiple future-token distributions in a single parallel step. Howev…
Temporal Alignment Guidance: On-Manifold Sampling in Diffusion Models
Youngrok Park, Hojung Jung, Sangmin Bae +1
Diffusion models have achieved remarkable success as generative models. However, even a well-trained model can accumulate errors throughout the generation process. These errors bec…
Mixture-of-Recursions: Learning Dynamic Recursive Depths for Adaptive Token-Level Computation
Sangmin Bae, Yujin Kim, Reza Bayat +8
Scaling language models unlocks impressive capabilities, but the accompanying computational and memory demands make both training and deployment expensive. Existing efficiency effo…
Revisiting Multi-Agent Debate as Test-Time Scaling: A Systematic Study of Conditional Effectiveness
Yongjin Yang, Euiin Yi, Jongwoo Ko +3
The remarkable growth in large language model (LLM) capabilities has spurred exploration into multi-agent systems, with debate frameworks emerging as a promising avenue for enhance…
DistiLLM-2: A Contrastive Approach Boosts the Distillation of LLMs
Jongwoo Ko, Tianyi Chen, Sungnyun Kim +4
Despite the success of distillation in large language models (LLMs), most prior work applies identical loss functions to both teacher- and student-generated data. These strategies…
MoHAVE: Mixture of Hierarchical Audio-Visual Experts for Robust Speech Recognition
Sungnyun Kim, Kangwook Jang, Sangmin Bae +2
Audio-visual speech recognition (AVSR) has become critical for enhancing speech recognition in noisy environments by integrating both auditory and visual modalities. However, exist…
Automated Filtering of Human Feedback Data for Aligning Text-to-Image Diffusion Models
Yongjin Yang, Sihyeon Kim, Hojung Jung +4
Fine-tuning text-to-image diffusion models with human feedback is an effective method for aligning model behavior with human intentions. However, this alignment process often suffe…
VACoDe: Visual Augmented Contrastive Decoding
Sihyeon Kim, Boryeong Cho, Sangmin Bae +2
Despite the astonishing performance of recent Large Vision-Language Models (LVLMs), these models often generate inaccurate responses. To address this issue, previous studies have f…
Learning Video Temporal Dynamics with Cross-Modal Attention for Robust Audio-Visual Speech Recognition
Sungnyun Kim, Kangwook Jang, Sangmin Bae +2
Audio-visual speech recognition (AVSR) aims to transcribe human speech using both audio and video modalities. In practical environments with noise-corrupted audio, the role of vide…
Fine-tuned In-Context Learning Transformers are Excellent Tabular Data Classifiers
Felix den Breejen, Sangmin Bae, Stephen Cha +1
The recently introduced TabPFN pretrains an In-Context Learning (ICL) transformer on synthetic data to perform tabular data classification. In this work, we extend TabPFN to the fi…
DistiLLM: Towards Streamlined Distillation for Large Language Models
Jongwoo Ko, Sungnyun Kim, Tianyi Chen +1
Knowledge distillation (KD) is widely used for compressing a teacher model to a smaller student model, reducing its inference cost and memory footprint while preserving model capab…
Fine-Tuning the Retrieval Mechanism for Tabular Deep Learning
Felix den Breejen, Sangmin Bae, Stephen Cha +3
While interests in tabular deep learning has significantly grown, conventional tree-based models still outperform deep learning methods. To narrow this performance gap, we explore…
Towards Difficulty-Agnostic Efficient Transfer Learning for Vision-Language Models
Yongjin Yang, Jongwoo Ko, Se-Young Yun
Vision-language models (VLMs) like CLIP have demonstrated remarkable applicability across a variety of downstream tasks, including zero-shot image classification. Recently, the use…
Fine tuning Pre trained Models for Robustness Under Noisy Labels
Sumyeong Ahn, Sihyeon Kim, Jongwoo Ko +1
The presence of noisy labels in a training dataset can significantly impact the performance of machine learning models. To tackle this issue, researchers have explored methods for…
NASH: A Simple Unified Framework of Structured Pruning for Accelerating Encoder-Decoder Language Models
Jongwoo Ko, Seungjoon Park, Yujin Kim +4
Structured pruning methods have proven effective in reducing the model size and accelerating inference speed in various network architectures such as Transformers. Despite the vers…
Fast and Robust Early-Exiting Framework for Autoregressive Language Models with Synchronized Parallel Decoding
Sangmin Bae, Jongwoo Ko, Hwanjun Song +1
To tackle the high inference latency exhibited by autoregressive language models, previous studies have proposed an early-exiting framework that allocates adaptive computation path…