10 papers
Adaptive Sampling and Clipping for Private Worst-Case Group Optimization
Max Cairney-Leeming, Amartya Sanyal, Christoph H. Lampert
A central requirement for the acceptance of machine learning methods for human-centric tasks is that they should be fair, in the sense that they should work comparably well for ind…
LoRA and Privacy: When Random Projections Help (and When They Don't)
Yaxi Hu, Johanna Düngler, Bernhard Schölkopf +1
We introduce the (Wishart) projection mechanism, a randomized map of the form with and study its differential privacy properties. For ve…
Learning in an Echo Chamber: Online Learning with Replay Adversary
Daniil Dmitriev, Harald Eskelund Franck, Carolin Heinzler +1
As machine learning systems increasingly train on self-annotated data, they risk reinforcing errors and becoming echo chambers of their own beliefs. We model this phenomenon by int…
Online Learning and Unlearning
Yaxi Hu, Bernhard Schölkopf, Amartya Sanyal
We formalize the problem of online learning-unlearning, where a model is updated sequentially in an online setting while accommodating unlearning requests between updates. After a…
Differentially Private Steering for Large Language Model Alignment
Anmol Goel, Yaxi Hu, Iryna Gurevych +1
Aligning Large Language Models (LLMs) with human values and away from undesirable behaviors (such as hallucination) has become increasingly important. Recently, steering LLMs towar…
On the Growth of Mistakes in Differentially Private Online Learning: A Lower Bound Perspective
Daniil Dmitriev, Kristóf Szabó, Amartya Sanyal
In this paper, we provide lower bounds for Differentially Private (DP) Online Learning algorithms. Our result shows that, for a broad class of -DP online algorith…