1 citations · 1 across the 8 of their papers we have counts for
9 papers
From Privacy to Generalization: Linear Max-Information Bounds for Differentially Private Learning Algorithms
Christoph H. Lampert, Max Cairney-Leeming, Hossein Zakerinia
Understanding the relationship between generalization and privacy remains a challenge in modern machine learning theory, particularly for deep networks that are trained by variants…
Beyond Square Roots: A Memory-Efficient Explicit Factorization for Multi-Epoch Private Learning
Nikita P. Kalinin, Aki Rehn, Joel Daniel Andersson +2
Correlated-noise mechanisms are among the most promising approaches for improving the utility of differentially private model training, but rigorous guarantees require explicit, an…
Optimization Risk Bounds for Kolmogorov-Arnold Networks Trained by DP-SGD with Correlated Noise
Puyu Wang, Jan Schuchardt, Nikita Kalinin +5
The theoretical understanding of differentially private stochastic gradient descent (DP-SGD) with temporally correlated noise remains limited, particularly for non-convex neural ne…
Sink vs. diagonal patterns as mechanisms for attention switch and oversmoothing prevention
Peter Súkeník, Cristina López Amado, Christoph H. Lampert +1
This paper studies the role of sinks and diagonal patterns as attention switch and anti-oversmoothing mechanisms. We analyze geometric conditions under which sinks can be represent…
DP-λCGD: Efficient Noise Correlation for Differentially Private Model Training
Nikita P. Kalinin, Ryan McKenna, Rasmus Pagh +1
Differentially private stochastic gradient descent (DP-SGD) is the gold standard for training machine learning models with formal differential privacy guarantees. Several recent ex…
Matrix Factorization for Practical Continual Mean Estimation Under User-Level Differential Privacy
Nikita P. Kalinin, Ali Najar, Valentin Roth +1
We study continual mean estimation, where data vectors arrive sequentially and the goal is to maintain accurate estimates of the running mean. We address this problem under user-le…
Back to Square Roots: An Optimal Bound on the Matrix Factorization Error for Multi-Epoch Differentially Private SGD
Nikita P. Kalinin, Ryan McKenna, Jalaj Upadhyay +1
Matrix factorization mechanisms for differentially private training have emerged as a promising approach to improve model utility under privacy constraints. In practical settings,…
Continual Release Moment Estimation with Differential Privacy
Nikita P. Kalinin, Jalaj Upadhyay, Christoph H. Lampert
We propose Joint Moment Estimation (JME), a method for continually and privately estimating both the first and second moments of data with reduced noise compared to naive approache…
Banded Square Root Matrix Factorization for Differentially Private Model Training
Nikita P. Kalinin, Christoph Lampert
Current state-of-the-art methods for differentially private model training are based on matrix factorization techniques. However, these methods suffer from high computational overh…