4 papers
FIRE: Frobenius-Isometry Reinitialization for Balancing the Stability-Plasticity Tradeoff
Isaac Han, Sangyeon Park, Seungwon Oh +3
Deep neural networks trained on nonstationary data must balance stability (i.e., retaining prior knowledge) and plasticity (i.e., adapting to new tasks). Standard reinitialization…
PREFAB: PREFerence-based Affective Modeling for Low-Budget Self-Annotation
Jaeyoung Moon, Youjin Choi, Yucheon Park +3
Self-annotation is the gold standard for collecting affective state labels in affective computing. Existing methods typically rely on full annotation, requiring users to continuous…
Recovering Plasticity of Neural Networks via Soft Weight Rescaling
Seungwon Oh, Sangyeon Park, Isaac Han +1
Recent studies have shown that as training progresses, neural networks gradually lose their capacity to learn new information, a phenomenon known as plasticity loss. An unbounded w…
Activation by Interval-wise Dropout: A Simple Way to Prevent Neural Networks from Plasticity Loss
Sangyeon Park, Isaac Han, Seungwon Oh +1
Plasticity loss, a critical challenge in neural network training, limits a model's ability to adapt to new tasks or shifts in data distribution. This paper introduces AID (Activati…