113 citations · 203 across the 21 of their papers we have counts for
12 papers · 1 filter
Continual Learning via Self-Probe Gradients
Dongkyu Cho, Rumi Chunara, Sungmin Cha
Adapting pretrained models to new data can cause catastrophic forgetting of previously learned behavior. When only a few past samples remain, they give continual learning methods s…
Why Knowledge Distillation Works in Generative Models: A Minimal Working Explanation
Sungmin Cha, Kyunghyun Cho
Knowledge distillation (KD) is a core component in the training and deployment of modern generative models, particularly large language models (LLMs). While its empirical benefits…
Forget Forgetting: Continual Learning in a World of Abundant Memory
Dongkyu Cho, Taesup Moon, Rumi Chunara +2
Continual learning (CL) has traditionally focused on minimizing exemplar memory, a constraint often misaligned with modern systems where GPU time, not storage, is the primary bottl…
Towards Robust and Parameter-Efficient Knowledge Unlearning for LLMs
Sungmin Cha, Sungjun Cho, Dasol Hwang +1
Large Language Models (LLMs) have demonstrated strong reasoning and memorization capabilities via pretraining on massive textual corpora. However, this poses risk of privacy and co…
Hyperparameters in Continual Learning: A Reality Check
Sungmin Cha, Kyunghyun Cho
Continual learning (CL) aims to train a model on a sequence of tasks (i.e., a CL scenario) while balancing the trade-off between plasticity (learning new tasks) and stability (reta…
Regularizing with Pseudo-Negatives for Continual Self-Supervised Learning
Sungmin Cha, Kyunghyun Cho, Taesup Moon
We introduce a novel Pseudo-Negative Regularization (PNR) framework for effective continual self-supervised learning (CSSL). Our PNR leverages pseudo-negatives obtained through mod…
Learning to Unlearn: Instance-wise Unlearning for Pre-trained Classifiers
Sungmin Cha, Sungjun Cho, Dasol Hwang +3
Since the recent advent of regulations for data protection (e.g., the General Data Protection Regulation), there has been increasing demand in deleting information learned from sen…
Towards Diverse Evaluation of Class Incremental Learning: A Representation Learning Perspective
Sungmin Cha, Jihwan Kwak, Dongsub Shim +4
Class incremental learning (CIL) algorithms aim to continually learn new object classes from incrementally arriving data while not forgetting past learned classes. The common evalu…
Supervised Neural Discrete Universal Denoiser for Adaptive Denoising
Sungmin Cha, Seonwoo Min, Sungroh Yoon +1
We improve the recently developed Neural DUDE, a neural network-based adaptive discrete denoiser, by combining it with the supervised learning framework. Namely, we make the superv…
CPR: Classifier-Projection Regularization for Continual Learning
Sungmin Cha, Hsiang Hsu, Taebaek Hwang +2
We propose a general, yet simple patch that can be applied to existing regularization-based continual learning methods called classifier-projection regularization (CPR). Inspired b…
Continual Learning with Node-Importance based Adaptive Group Sparse Regularization
Sangwon Jung, Hongjoon Ahn, Sungmin Cha +1
We propose a novel regularization-based continual learning method, dubbed as Adaptive Group Sparsity based Continual Learning (AGS-CL), using two group sparsity-based penalties. Ou…
Uncertainty-based Continual Learning with Adaptive Regularization
Hongjoon Ahn, Sungmin Cha, Donggyu Lee +1
We introduce a new neural network-based continual learning algorithm, dubbed as Uncertainty-regularized Continual Learning (UCL), which builds on traditional Bayesian online learni…