8 papers
Techniques for Peak Memory Reduction for LoRA Fine-tuning of LLMs on Edge Devices
Hassan Dbouk, Matthias Reisser, Prathamesh Mandke +2
Fine-tuning of Large Language Models (LLMs) using Low-Rank Adaptation (LoRA) on an end-user's data offers personalized experiences while keeping data private, but faces severe memo…
On Adaptivity in Zeroth-Order Optimization
Hassan Dbouk, Nidham Gazagnadou, Matthias Reisser +1
We investigate the effectiveness of adaptive zeroth-order (ZO) optimization for memory-constrained fine-tuning of large language models (LLMs). Contrary to prior claims, we show th…
Private PoEtry: Private In-Context Learning via Product of Experts
Rob Romijnders, Mohammad Mahdi Derakhshani, Jonathan Petit +3
In-context learning (ICL) enables Large Language Models (LLMs) to adapt to new tasks with only a small set of examples at inference time, thereby avoiding task-specific fine-tuning…
Stable Diffusion-based Data Augmentation for Federated Learning with Non-IID Data
Mahdi Morafah, Matthias Reisser, Bill Lin +1
The proliferation of edge devices has brought Federated Learning (FL) to the forefront as a promising paradigm for decentralized and collaborative model training while preserving t…
A Mutual Information Perspective on Federated Contrastive Learning
Christos Louizos, Matthias Reisser, Denis Korzhenkov
We investigate contrastive learning in the federated setting through the lens of SimCLR and multi-view mutual information maximization. In doing so, we uncover a connection between…
DNA: Differentially private Neural Augmentation for contact tracing
Rob Romijnders, Christos Louizos, Yuki M. Asano +1
The COVID19 pandemic had enormous economic and societal consequences. Contact tracing is an effective way to reduce infection rates by detecting potential virus carriers early. How…