1 citations · 1 across the 7 of their papers we have counts for
6 papers · 1 filter
Optimizer choice matters for the emergence of Neural Collapse
Jim Zhao, Tin Sum Cheng, Wojciech Masarczyk +1
Neural Collapse (NC) refers to the emergence of highly symmetric geometric structures in the representations of deep neural networks during the terminal phase of training. Despite…
Spectral Analysis of Molecular Features: When Richer Features Do Not Guarantee Better Generalization
Asma Jamali, Tin Sum Cheng, Rodrigo A. Vargas-Hernández
The spectral properties of feature embeddings offer critical insights into model generalization and representation quality. While deep learning models are widely used for molecular…
Unpacking Softmax: How Temperature Drives Representation Collapse, Compression, and Generalization
Wojciech Masarczyk, Mateusz Ostaszewski, Tin Sum Cheng +3
The softmax function is a fundamental building block of deep neural networks, commonly used to define output distributions in classification tasks or attention weights in transform…
A Comprehensive Analysis on the Learning Curve in Kernel Ridge Regression
Tin Sum Cheng, Aurelien Lucchi, Anastasis Kratsios +1
This paper conducts a comprehensive study of the learning curves of kernel ridge regression (KRR) under minimal assumptions. Our contributions are three-fold: 1) we analyze the rol…
Characterizing Overfitting in Kernel Ridgeless Regression Through the Eigenspectrum
Tin Sum Cheng, Aurelien Lucchi, Anastasis Kratsios +1
We derive new bounds for the condition number of kernel matrices, which we then use to enhance existing non-asymptotic test error bounds for kernel ridgeless regression (KRR) in th…
A Theoretical Analysis of the Test Error of Finite-Rank Kernel Ridge Regression
Tin Sum Cheng, Aurelien Lucchi, Ivan Dokmanić +2
Existing statistical learning guarantees for general kernel regressors often yield loose bounds when used with finite-rank kernels. Yet, finite-rank kernels naturally appear in sev…