1 citations · 1 across the 3 of their papers we have counts for
6 papers
Forget Forgetting: Continual Learning in a World of Abundant Memory
Dongkyu Cho, Taesup Moon, Rumi Chunara +2
Continual learning (CL) has traditionally focused on minimizing exemplar memory, a constraint often misaligned with modern systems where GPU time, not storage, is the primary bottl…
Why Knowledge Distillation Works in Generative Models: A Minimal Working Explanation
Sungmin Cha, Kyunghyun Cho
Knowledge distillation (KD) is a core component in the training and deployment of modern generative models, particularly large language models (LLMs). While its empirical benefits…
Hyperparameters in Continual Learning: A Reality Check
Sungmin Cha, Kyunghyun Cho
Continual learning (CL) aims to train a model on a sequence of tasks (i.e., a CL scenario) while balancing the trade-off between plasticity (learning new tasks) and stability (reta…
Reference-Specific Unlearning Metrics Can Hide the Truth: A Reality Check
Sungjun Cho, Dasol Hwang, Frederic Sala +3
Current unlearning metrics for generative models evaluate success based on reference responses or classifier outputs rather than assessing the core objective: whether the unlearned…
Why Alignment Must Precede Distillation: A Minimal Working Explanation
Sungmin Cha, Kyunghyun Cho
For efficiency, preference alignment is often performed on compact, knowledge-distilled (KD) models. We argue this common practice introduces a significant limitation by overlookin…
Regularizing with Pseudo-Negatives for Continual Self-Supervised Learning
Sungmin Cha, Kyunghyun Cho, Taesup Moon
We introduce a novel Pseudo-Negative Regularization (PNR) framework for effective continual self-supervised learning (CSSL). Our PNR leverages pseudo-negatives obtained through mod…