17 citations · 39 across the 35 of their papers we have counts for
24 papers · 1 filter
Extracting Forgotten Prompts from Targeted Unlearned Models
Au Ashley Hoi-Ting, Meghdad Kurmanji, William F. Shen +2
Recent unlearning methods (e.g. NPO, DPO, LUNAR) make use of refusal alignment to suppress forgotten data. However, it has been shown that refusal responses might leave traces of u…
Flower Hub: A Reproducible Benchmarking Platform for Federated Learning in Simulation and Deployment
Yan Gao, Mohammad Naseri, Javier Fernandez-Marques +19
Federated learning (FL) has emerged as a key approach for training models across decentralized data, yet benchmarking in FL remains difficult to reproduce, compare, and extend. Exi…
The Red Queen Gödel Machine: Co-Evolving Agents and Their Evaluators
Alex Iacob, Andrej Jovanović, William F. Shen +12
Self-improving agents are state-of-the-art on agentic coding benchmarks, yet their search methods assume a stationary evaluation criterion. This ignores a central feature of evolut…
FoMoE: Breaking the Full-Replica Barrier with a Federation of MoEs
Lorenzo Sani, Zeyu Cao, Meghdad Kurmanji +5
Pre-training Large Language Models (LLMs) typically demands large-scale infrastructure with tightly coupled hardware accelerators. Mixture-of-Experts (MoEs) architectures partially…
Beyond Scaling: Agents Are Heading to the Edge
Chunlin Tian, Dongqi Cai, Wanru Zhao +1
The bottleneck of useful agentic intelligence has shifted from compressing world knowledge into a single model to executing a coordinated system. This position paper argues that pe…
Task-Centric Personalized Federated Fine-Tuning of Language Models
Gabriel U. Talasso, Meghdad Kurmanji, Allan M. de Souza +2
Federated Learning (FL) has emerged as a promising technique for training language models on distributed and private datasets of diverse tasks. However, aggregating models trained…
-FUM: Federated Unlearning via min--max and -divergence
Radmehr Karimian, Amirhossein Bagheri, Meghdad Kurmanji +2
Federated learning (FL) enables collaborative training while keeping raw data on clients. Deletion requests and the discovery of poisoned data create a need to remove selected cont…
LoRDO: Distributed Low-Rank Optimization with Infrequent Communication
Andrej Jovanović, Alex Iacob, Mher Safaryan +6
Distributed training of foundation models via is limited by interconnect bandwidth. While infrequent communication strategies reduce synchronization frequency, they…
MT-DAO: Multi-Timescale Distributed Adaptive Optimizers with Local Updates
Alex Iacob, Andrej Jovanovic, Mher Safaryan +6
Training large models with distributed data parallelism (DDP) requires frequent communication of gradients across workers, which can saturate bandwidth. Infrequent communication st…
AbbIE: Autoregressive Block-Based Iterative Encoder for Efficient Sequence Modeling
Preslav Aleksandrov, Meghdad Kurmanji, Fernando Garcia Redondo +7
We introduce the Autoregressive Block-Based Iterative Encoder (AbbIE), a novel recursive generalization of the encoder-only Transformer architecture, which achieves better perplexi…
DES-LOC: Desynced Low Communication Adaptive Optimizers for Training Foundation Models
Alex Iacob, Lorenzo Sani, Mher Safaryan +8
Scaling foundation model training with Distributed Data Parallel (DDP) methods is bandwidth-limited. Existing infrequent communication methods like Local SGD were designed to synch…
Editing as Unlearning: Are Knowledge Editing Methods Strong Baselines for Large Language Model Unlearning?
Zexi Li, Xiangzhu Wang, William F. Shen +5
Large language Model (LLM) unlearning, i.e., selectively removing information from LLMs, is vital for responsible model deployment. Differently, LLM knowledge editing aims to modif…
SparsyFed: Sparse Adaptive Federated Training
Adriano Guastella, Lorenzo Sani, Alex Iacob +3
Sparse training is often adopted in cross-device federated learning (FL) environments where constrained devices collaboratively train a machine learning model on private data by ex…
Exploring Code Language Models for Automated HLS-based Hardware Generation: Benchmark, Infrastructure and Analysis
Jiahao Gai, Hao Mark Chen, Zhican Wang +4
Recent advances in code generation have illuminated the potential of employing large language models (LLMs) for general-purpose programming languages such as Python and C++, openin…
Position: Bridge the Gaps between Machine Unlearning and AI Regulation
Bill Marino, Meghdad Kurmanji, Nicholas D. Lane
The ''right to be forgotten'' and the data privacy laws that encode it have motivated machine unlearning since its earliest days. Now, some argue that an inbound wave of artificial…
LLM Unlearning via Neural Activation Redirection
William F. Shen, Xinchi Qiu, Meghdad Kurmanji +5
The ability to selectively remove knowledge from LLMs is highly desirable. However, existing methods often struggle with balancing unlearning efficacy and retain model utility, and…
Photon: Federated LLM Pre-Training
Lorenzo Sani, Alex Iacob, Zeyu Cao +8
Scaling large language models (LLMs) demands extensive data and computing resources, which are traditionally constrained to data centers by the high-bandwidth requirements of distr…
DEPT: Decoupled Embeddings for Pre-training Language Models
Alex Iacob, Lorenzo Sani, Meghdad Kurmanji +5
Language Model pre-training uses broad data mixtures to enhance performance across domains and languages. However, training on such heterogeneous text corpora requires extensive an…
How Data Inter-connectivity Shapes LLMs Unlearning: A Structural Unlearning Perspective
Xinchi Qiu, William F. Shen, Yihong Chen +4
While unlearning knowledge from large language models (LLMs) is receiving increasing attention, one important aspect remains unexplored. Existing approaches and benchmarks assume d…
Sheaf HyperNetworks for Personalized Federated Learning
Bao Nguyen, Lorenzo Sani, Xinchi Qiu +2
Graph hypernetworks (GHNs), constructed by combining graph neural networks (GNNs) with hypernetworks (HNs), leverage relational data across various domains such as neural architect…
Worldwide Federated Training of Language Models
Alex Iacob, Lorenzo Sani, Bill Marino +3
The reliance of language model training on massive amounts of computation and vast datasets scraped from potentially low-quality, copyrighted, or sensitive data has come into quest…
The Future of Large Language Model Pre-training is Federated
Lorenzo Sani, Alex Iacob, Zeyu Cao +8
Generative pre-trained large language models (LLMs) have demonstrated impressive performance over a wide range of tasks, thanks to the unprecedented amount of data they have been t…
Federated Learning Priorities Under the European Union Artificial Intelligence Act
Herbert Woisetschläger, Alexander Erben, Bill Marino +4
The age of AI regulation is upon us, with the European Union Artificial Intelligence Act (AI Act) leading the way. Our key inquiry is how this will affect Federated Learning (FL),…
FedGuCci: Making Local Models More Connected in Landscape for Federated Learning
Zexi Li, Jie Lin, Zhiqi Li +5
Federated learning (FL) involves multiple heterogeneous clients collaboratively training a global model via iterative local updates and model fusion. The generalization of FL's glo…