16 papers
LoRA Provides Differential Privacy by Design via Random Sketching
Saber Malekmohammadi, Golnoosh Farnadi
Low-rank adaptation of language models has been proposed to reduce the computational and memory overhead of fine-tuning pre-trained language models. LoRA incorporates trainable low…
Towards More Realistic Extraction Attacks: An Adversarial Perspective
Yash More, Prakhar Ganesh, Golnoosh Farnadi
Language models are prone to memorizing their training data, making them vulnerable to extraction attacks. While existing research often examines isolated setups, such as a single…
Beyond the Safety Bundle: Auditing the Helpful and Harmless Dataset
Khaoula Chehbouni, Jonathan Colaço Carr, Yash More +2
In an effort to mitigate the harms of large language models (LLMs), learning from human feedback (LHF) has been used to steer LLMs towards outputs that are intended to be both less…
Differentially Private Clustered Federated Learning
Saber Malekmohammadi, Afaf Taik, Golnoosh Farnadi
Federated learning (FL), which is a decentralized machine learning (ML) approach, often incorporates differential privacy (DP) to provide rigorous data privacy guarantees. Previous…
What Secrets Do Your Manifolds Hold? Understanding the Local Geometry of Generative Models
Ahmed Imtiaz Humayun, Ibtihel Amara, Cristina Vasconcelos +7
Deep Generative Models are frequently used to learn continuous representations of complex data distributions using a finite number of samples. For any generative model, including p…
Embedding Cultural Diversity in Prototype-based Recommender Systems
Armin Moradi, Nicola Neophytou, Florian Carichon +1
Popularity bias in recommender systems can increase cultural overrepresentation by favoring norms from dominant cultures and marginalizing underrepresented groups. This issue is cr…