335 citations · 1.6k across the 27 of their papers we have counts for
28 papers
Power quotients of plactic-like monoids
Antoine Abram, Florent Hivert, James D. Mitchell +2
In this paper we describe the quotients of several plactic-like monoids by the least congruences containing the relations for for every generator .…
Techniques for Peak Memory Reduction for LoRA Fine-tuning of LLMs on Edge Devices
Hassan Dbouk, Matthias Reisser, Prathamesh Mandke +2
Fine-tuning of Large Language Models (LLMs) using Low-Rank Adaptation (LoRA) on an end-user's data offers personalized experiences while keeping data private, but faces severe memo…
On Adaptivity in Zeroth-Order Optimization
Hassan Dbouk, Nidham Gazagnadou, Matthias Reisser +1
We investigate the effectiveness of adaptive zeroth-order (ZO) optimization for memory-constrained fine-tuning of large language models (LLMs). Contrary to prior claims, we show th…
Private PoEtry: Private In-Context Learning via Product of Experts
Rob Romijnders, Mohammad Mahdi Derakhshani, Jonathan Petit +3
In-context learning (ICL) enables Large Language Models (LLMs) to adapt to new tasks with only a small set of examples at inference time, thereby avoiding task-specific fine-tuning…
Stable Diffusion-based Data Augmentation for Federated Learning with Non-IID Data
Mahdi Morafah, Matthias Reisser, Bill Lin +1
The proliferation of edge devices has brought Federated Learning (FL) to the forefront as a promising paradigm for decentralized and collaborative model training while preserving t…
A Mutual Information Perspective on Federated Contrastive Learning
Christos Louizos, Matthias Reisser, Denis Korzhenkov
We investigate contrastive learning in the federated setting through the lens of SimCLR and multi-view mutual information maximization. In doing so, we uncover a connection between…
DNA: Differentially private Neural Augmentation for contact tracing
Rob Romijnders, Christos Louizos, Yuki M. Asano +1
The COVID19 pandemic had enormous economic and societal consequences. Contact tracing is an effective way to reduce infection rates by detecting potential virus carriers early. How…
InterroGate: Learning to Share, Specialize, and Prune Representations for Multi-task Learning
Babak Ehteshami Bejnordi, Gaurav Kumar, Amelie Royer +3
Jointly learning multiple tasks with a unified model can improve accuracy and data efficiency, but it faces the challenge of task interference, where optimizing one task objective…
Protect Your Score: Contact Tracing With Differential Privacy Guarantees
Rob Romijnders, Christos Louizos, Yuki M. Asano +1
The pandemic in 2020 and 2021 had enormous economic and societal consequences, and studies show that contact tracing algorithms can be key in the early containment of the virus. Wh…
Hyperparameter Optimization through Neural Network Partitioning
Bruno Mlodozeniec, Matthias Reisser, Christos Louizos
Well-tuned hyperparameters are crucial for obtaining good generalization behavior in neural networks. They can enforce appropriate inductive biases, regularize the model and improv…
Quantization Robust Federated Learning for Efficient Inference on Heterogeneous Devices
Kartik Gupta, Marios Fournarakis, Matthias Reisser +2
Federated Learning (FL) is a machine learning paradigm to distributively learn machine learning models from decentralized data that remains on-device. Despite the success of standa…
DP-REC: Private & Communication-Efficient Federated Learning
Aleksei Triastcyn, Matthias Reisser, Christos Louizos
Privacy and communication efficiency are important challenges in federated training of neural networks, and combining them is still an open problem. In this work, we develop a meth…
An Expectation-Maximization Perspective on Federated Learning
Christos Louizos, Matthias Reisser, Joseph Soriaga +1
Federated learning describes the distributed training of models across multiple clients while keeping the data private on-device. In this work, we view the server-orchestrated fede…
Federated Mixture of Experts
Matthias Reisser, Christos Louizos, Efstratios Gavves +1
Federated learning (FL) has emerged as the predominant approach for collaborative training of neural network models across multiple users, without the need to gather the data at a…
Federated Learning of User Verification Models Without Sharing Embeddings
Hossein Hosseini, Hyunsin Park, Sungrack Yun +3
We consider the problem of training User Verification (UV) models in federated setting, where each user has access to the data of only one class and user embeddings cannot be share…
Federated Learning of User Authentication Models
Hossein Hosseini, Sungrack Yun, Hyunsin Park +3
Machine learning-based User Authentication (UA) models have been widely deployed in smart devices. UA models are trained to map input data of different users to highly separable em…
Bayesian Bits: Unifying Quantization and Pruning
Mart van Baalen, Christos Louizos, Markus Nagel +4
We introduce Bayesian Bits, a practical method for joint mixed precision quantization and pruning through gradient based optimization. Bayesian Bits employs a novel decomposition o…
Up or Down? Adaptive Rounding for Post-Training Quantization
Markus Nagel, Rana Ali Amjad, Mart van Baalen +2
When quantizing neural networks, assigning each floating-point weight to its nearest fixed-point value is the predominant approach. We find that, perhaps surprisingly, this is not…
Gradient Regularization for Quantization Robustness
Milad Alizadeh, Arash Behboodi, Mart van Baalen +3
We analyze the effect of quantizing weights and activations of neural networks on their loss and derive a simple regularization scheme that improves robustness against post-trainin…
The Functional Neural Process
Christos Louizos, Xiahan Shi, Klamer Schutte +1
We present a new family of exchangeable stochastic processes, the Functional Neural Processes (FNPs). FNPs model distributions over functions by learning a graph of dependencies on…
DIVA: Domain Invariant Variational Autoencoders
Maximilian Ilse, Jakub M. Tomczak, Christos Louizos +1
We consider the problem of domain generalization, namely, how to learn representations given data from a set of domains that generalize to data from a previously unseen domain. We…
Relaxed Quantization for Discretized Neural Networks
Christos Louizos, Matthias Reisser, Tijmen Blankevoort +2
Neural network quantization has become an important research area due to its great impact on deployment of large models on resource constrained devices. In order to train networks…
Learning Sparse Neural Networks through Regularization
Christos Louizos, Max Welling, Diederik P. Kingma
We propose a practical method for norm regularization for neural networks: pruning the network during training by encouraging weights to become exactly zero. Such regularizat…
Causal Effect Inference with Deep Latent-Variable Models
Christos Louizos, Uri Shalit, Joris Mooij +3
Learning individual-level causal effects from observational data, such as inferring the most effective medication for a specific patient, is a problem of growing importance for pol…