8 papers
Self-Supervised Learning by Curvature Alignment
Benyamin Ghojogh, M. Hadi Sepanj, Paul Fieguth
Self-supervised learning (SSL) has recently advanced through non-contrastive methods that couple an invariance term with variance, covariance, or redundancy-reduction penalties. Wh…
Kernel VICReg for Self-Supervised Learning in Reproducing Kernel Hilbert Space
M. Hadi Sepanj, Benyamin Ghojogh, Saed Moradi +1
Self-supervised learning (SSL) has emerged as a powerful paradigm for representation learning by optimizing geometric objectives, such as invariance to augmentations, variance pres…
Pre-train to Gain: Robust Learning Without Clean Labels
David Szczecina, Nicholas Pellegrino, Paul Fieguth
Training deep networks with noisy labels leads to poor generalization and degraded accuracy due to overfitting to label noise. Existing approaches for learning with noisy labels of…
Effects of Initialization Biases on Deep Neural Network Training Dynamics
Nicholas Pellegrino, David Szczecina, Paul W. Fieguth
Untrained large neural networks, just after random initialization, tend to favour a small subset of classes, assigning high predicted probabilities to these few classes and approxi…
Hard Samples, Bad Labels: Robust Loss Functions That Know When to Back Off
Nicholas Pellegrino, David Szczecina, Paul Fieguth
Incorrectly labelled training data are frustratingly ubiquitous in both benchmark and specially curated datasets. Such mislabelling clearly adversely affects the performance and ge…
Self-Supervised Learning Using Nonlinear Dependence
M. Hadi Sepanj, Benyamin Ghojogh, Paul Fieguth
Self-supervised learning has gained significant attention in contemporary applications, particularly due to the scarcity of labeled data. While existing SSL methodologies primarily…