5 papers · 1 filter
Continual Learning via Self-Probe Gradients
Dongkyu Cho, Rumi Chunara, Sungmin Cha
Adapting pretrained models to new data can cause catastrophic forgetting of previously learned behavior. When only a few past samples remain, they give continual learning methods s…
Forgetting in Language Models: Capacity, Optimization, and Self-Generated Replay
Martin Marek, Dongkyu Cho, Shikai Qiu +3
Models trained on a new task typically degrade on prior tasks, a phenomenon known as forgetting. Traditionally, mitigating forgetting has required replaying stored exemplars from p…
Expert-guided Clinical Text Augmentation via Query-Based Model Collaboration
Dongkyu Cho, Miao Zhang, Rumi Chunara
Data augmentation is a widely used strategy to improve model robustness and generalization by enriching training datasets with synthetic examples. While large language models (LLMs…
Dealing with the Evil Twins: Improving Random Augmentation by Addressing Catastrophic Forgetting of Diverse Augmentations
Dongkyu Cho, Rumi Chunara
Data augmentation is a promising tool for enhancing out-of-distribution generalization, where the key is to produce diverse, challenging variations of the source domain via costly…
Forget Forgetting: Continual Learning in a World of Abundant Memory
Dongkyu Cho, Taesup Moon, Rumi Chunara +2
Continual learning (CL) has traditionally focused on minimizing exemplar memory, a constraint often misaligned with modern systems where GPU time, not storage, is the primary bottl…