3 papers
stat.ML2025
Beyond Universal Approximation Theorems: Algorithmic Uniform Approximation by Neural Networks Trained with Noisy Data
Anastasis Kratsios, Tin Sum Cheng, Daniel Roy
At its core, machine learning seeks to train models that reliably generalize beyond noisy observations; however, the theoretical vacuum in which state-of-the-art universal approxim…
stat.ML2025
Sharp Generalization Bounds for Foundation Models with Asymmetric Randomized Low-Rank Adapters
Anastasis Kratsios, Tin Sum Cheng, Aurelien Lucchi +1
Low-Rank Adaptation (LoRA) has emerged as a widely adopted parameter-efficient fine-tuning (PEFT) technique for foundation models. Recent work has highlighted an inherent asymmetry…
cs.LG2025
Unpacking Softmax: How Temperature Drives Representation Collapse, Compression, and Generalization
Wojciech Masarczyk, Mateusz Ostaszewski, Tin Sum Cheng +3
The softmax function is a fundamental building block of deep neural networks, commonly used to define output distributions in classification tasks or attention weights in transform…