9 papers · 1 filter
The Neural Tangent Kernel for Classification
Jonathan Plenk, Sergio Calvo-Ordonez, Alvaro Cartea +3
In wide neural networks, the Neural Tangent Kernel (NTK) remains approximately constant during training, providing a powerful theoretical tool for studying training dynamics, gener…
Fast Adversarial Attacks with Gradient Prediction
Kamil Ciosek, Aleksandr V. Petrov, Nicolò Felicioni +1
Generating adversarial examples at scale is a core primitive for robustness evaluation, adversarial training, and red-teaming, yet even "fast" attacks such as FGSM remain throughpu…
The Minimax Rate of Perturbed Second-Order Calibration
Kamil Ciosek, Banafsheh Rafiee, Sina Ghiassian +1
Second-order calibration error quantifies how closely a higher-order predictor's epistemic-uncertainty estimate matches the conditional variance of the label probability on its lev…
A Bayesian Information-Theoretic Approach to Data Attribution
Dharmesh Tailor, Nicolò Felicioni, Kamil Ciosek
Training Data Attribution (TDA) seeks to trace model predictions back to influential training examples, enhancing interpretability and safety. We formulate TDA as a Bayesian inform…
Richer Bayesian Last Layers with Subsampled NTK Features
Sergio Calvo-Ordoñez, Jonathan Plenk, Richard Bergna +4
Bayesian Last Layers (BLLs) provide a convenient and computationally efficient way to estimate uncertainty in neural networks. However, they underestimate epistemic uncertainty bec…
When Embedding Models Meet: Procrustes Bounds and Applications
Lucas Maystre, Alvaro Ortega Gonzalez, Charles Park +4
Embedding models trained separately on similar data often produce representations that encode stable information but are not directly interchangeable. This lack of interoperability…
A Gaussian Process View on Observation Noise and Initialization in Wide Neural Networks
Sergio Calvo-Ordoñez, Jonathan Plenk, Richard Bergna +4
Performing gradient descent in a wide neural network is equivalent to computing the posterior mean of a Gaussian Process with the Neural Tangent Kernel (NTK-GP), for a specific pri…
Impatient Bandits: Optimizing for the Long-Term Without Delay
Kelly W. Zhang, Thomas Baldwin-McDonald, Kamil Ciosek +2
Increasingly, recommender systems are tasked with improving users' long-term satisfaction. In this context, we study a content exploration task, which we formalize as a bandit prob…
Epistemic Uncertainty and Observation Noise with the Neural Tangent Kernel
Sergio Calvo-Ordoñez, Konstantina Palla, Kamil Ciosek
Recent work has shown that training wide neural networks with gradient descent is formally equivalent to computing the mean of the posterior distribution in a Gaussian Process (GP)…