14 papers
MoSE: Mixture of Slimmable Experts for Efficient and Adaptive Language Models
Nurbek Tastan, Stefanos Laskaridis, Karthik Nandakumar +1
Mixture-of-Experts (MoE) models scale large language models efficiently by sparsely activating experts, but once an expert is selected, it is executed fully. Hence, the trade-off b…
PreLort: Prefix-Nested LoRA for Federated Fine-Tuning under Rank Heterogeneity
Muhammad Waseem, Nurbek Tastan, Andrej Jovanovic +4
Federated fine-tuning of large language models using parameter-efficient methods such as LoRA enables privacy-preserving adaptation of foundation models. Heterogeneous hardware res…
Response-Conditioned Parallel-to-Sequential Orchestration for Multi-Agent Systems
Nurbek Tastan, Alex Iacob, Lorenzo Sani +4
Multi-agent systems can solve complex tasks through collaboration between multiple Large Language Model agents. Existing collaboration frameworks typically operate in either a para…
Data-Free Contribution Estimation in Federated Learning using Gradient von Neumann Entropy
Asim Ukaye, Mubarak Abdu-Aguye, Nurbek Tastan +1
Client contribution estimation in Federated Learning is necessary for identifying clients' importance and for providing fair rewards. Current methods often rely on server-side vali…
SPDMark: Selective Parameter Displacement for Robust Video Watermarking
Samar Fares, Nurbek Tastan, Karthik Nandakumar
The advent of high-quality video generation models has amplified the need for robust watermarking schemes that can be used to reliably detect and track the provenance of generated…
MOLM: Mixture of LoRA Markers
Samar Fares, Nurbek Tastan, Noor Hussein +1
Generative models can generate photorealistic images at scale. This raises urgent concerns about the ability to detect synthetically generated images and attribute these images to…