5 papers
Memorization in Fine-Tuned Large Language Models
Danil Savine
This study investigates the mechanisms and factors influencing memorization in fine-tuned large language models (LLMs), with a focus on the medical domain due to its privacy-sensit…
Unveiling the Role of Randomization in Multiclass Adversarial Classification: Insights from Graph Theory
Lucas Gnecco-Heredia, Matteo Sammut, Muni Sreenivas Pydi +3
Randomization as a mean to improve the adversarial robustness of machine learning models has recently attracted significant attention. Unfortunately, much of the theoretical analys…
Memorization in Attention-only Transformers
Léo Dana, Muni Sreenivas Pydi, Yann Chevaleyre
Recent research has explored the memorization capacity of multi-head attention, but these findings are constrained by unrealistic limitations on the context size. We present a nove…
Differentially Private Gradient Flow based on the Sliced Wasserstein Distance
Ilana Sebag, Muni Sreenivas Pydi, Jean-Yves Franceschi +4
Safeguarding privacy in sensitive training data is paramount, particularly in the context of generative modeling. This can be achieved through either differentially private stochas…
Optimal Classification under Performative Distribution Shift
Edwige Cyffers, Muni Sreenivas Pydi, Jamal Atif +1
Performative learning addresses the increasingly pervasive situations in which algorithmic decisions may induce changes in the data distribution as a consequence of their public de…