7 papers
Self-Preservation in Language Models
Dongkyu Cho, Rumi Chunara, Ali Behrouz
Continual learning (CL) seeks to adapt pretrained models while preserving prior capabilities. Yet the tasks supplied for preservation cover only part of what a model knows, while f…
Continual Learning via Self-Probe Gradients
Dongkyu Cho, Rumi Chunara, Sungmin Cha
Adapting pretrained models to new data can cause catastrophic forgetting of previously learned behavior. When only a few past samples remain, they give continual learning methods s…
Forgetting in Language Models: Capacity, Optimization, and Self-Generated Replay
Martin Marek, Dongkyu Cho, Shikai Qiu +3
Models trained on a new task typically degrade on prior tasks, a phenomenon known as forgetting. Traditionally, mitigating forgetting has required replaying stored exemplars from p…
Correct Reasoning Paths Visit Shared Decision Pivots
Dongkyu Cho, Amy B. Z. Zhang, Bilel Fehri +4
Chain-of-thought (CoT) reasoning exposes the intermediate thinking process of large language models (LLMs), yet verifying those traces at scale remains unsolved. In response, we in…
Expert-guided Clinical Text Augmentation via Query-Based Model Collaboration
Dongkyu Cho, Miao Zhang, Rumi Chunara
Data augmentation is a widely used strategy to improve model robustness and generalization by enriching training datasets with synthetic examples. While large language models (LLMs…
Dealing with the Evil Twins: Improving Random Augmentation by Addressing Catastrophic Forgetting of Diverse Augmentations
Dongkyu Cho, Rumi Chunara
Data augmentation is a promising tool for enhancing out-of-distribution generalization, where the key is to produce diverse, challenging variations of the source domain via costly…
Forget Forgetting: Continual Learning in a World of Abundant Memory
Dongkyu Cho, Taesup Moon, Rumi Chunara +2
Continual learning (CL) has traditionally focused on minimizing exemplar memory, a constraint often misaligned with modern systems where GPU time, not storage, is the primary bottl…