7 papers
Mind the Spike: Mechanisms and Brittleness of Visual Massive Activations in Large Vision-Language Models
Jonas Ngnawé, Yann Pequignot, Sabyasachi Sahoo +3
Large vision-language models (LVLMs) inherit massive activations from their text-only bases: spikes where a few fixed hidden channels receive values thousands of times above the ty…
A Stochastic--Geometric Theory of Scaling Laws in Grokking
Róisín Luo, Christian Gagné, Jonas Ngnawé +2
Delayed generalization (\ie~grokking) refers to the phenomenon in which a neural network fits its training data early in training but only begins to generalize after a prolonged de…
Reliability-Gated Source Anchoring for Continual Test-Time Adaptation
Vikash Singh, Debargha Ganguly, Weicong Chen +8
Continual test-time adaptation (CTTA) updates a pretrained model online on an unlabeled, non-stationary stream while anchoring it to a frozen source checkpoint. This anchor is usef…
Optimization-Induced Dynamics of Lipschitz Continuity in Neural Networks
Róisín Luo, James McDermott, Christian Gagné +2
Lipschitz continuity characterizes the worst-case sensitivity of neural networks to small input perturbations; yet its dynamics (i.e. temporal evolution) during training remains un…
A Layer Selection Approach to Test Time Adaptation
Sabyasachi Sahoo, Mostafa ElAraby, Jonas Ngnawe +3
Test Time Adaptation (TTA) addresses the problem of distribution shift by adapting a pretrained model to a new domain during inference. When faced with challenging shifts, most met…
Generalizing across Temporal Domains with Koopman Operators
Qiuhao Zeng, Wei Wang, Fan Zhou +7
In the field of domain generalization, the task of constructing a predictive model capable of generalizing to a target domain without access to target data remains challenging. Thi…
Accumulating Knowledge for Lifelong Online Learning
Changjian Shui, Ihsen Hedhli, Christian Gagné
Lifelong learning can be viewed as a continuous transfer learning procedure over consecutive tasks, where learning a given task depends on accumulated knowledge --- the so-called k…