17 citations · 39 across the 35 of their papers we have counts for
15 papers · 1 filter
Learning to Read Out: Unembedding Dynamics in Language Model Pretraining
Matteo He, William F. Shen, Alex Iacob +3
When does a language model acquire a capability? Developmental studies usually answer this by tracking when models start producing the correct token. But that prediction passes thr…
Extracting Forgotten Prompts from Targeted Unlearned Models
Au Ashley Hoi-Ting, Meghdad Kurmanji, William F. Shen +2
Recent unlearning methods (e.g. NPO, DPO, LUNAR) make use of refusal alignment to suppress forgotten data. However, it has been shown that refusal responses might leave traces of u…
Sparse Readout Prism: Explaining Logit-Lens Scores in Features Instead of Tokens
Matteo He, William F. Shen, Xinchi Qiu +1
A language model's prediction of its next token develops across layers, and lens methods track this process by decoding intermediate hidden states into tokens. But a lens reading r…
Flower Hub: A Reproducible Benchmarking Platform for Federated Learning in Simulation and Deployment
Yan Gao, Mohammad Naseri, Javier Fernandez-Marques +19
Federated learning (FL) has emerged as a key approach for training models across decentralized data, yet benchmarking in FL remains difficult to reproduce, compare, and extend. Exi…
How LLM Task-Adaptation Reshapes Alignment: A Multi-dimensional Study of Behavioral and Representational Drift
James Elcock, William F. Shen, Xinchi Qiu +1
Post-training is a key mechanism for adapting large language models to downstream tasks. While prior work suggests that task adaptation can alter a model's pre-existing alignment,…
The Red Queen Gödel Machine: Co-Evolving Agents and Their Evaluators
Alex Iacob, Andrej Jovanović, William F. Shen +12
Self-improving agents are state-of-the-art on agentic coding benchmarks, yet their search methods assume a stationary evaluation criterion. This ignores a central feature of evolut…
FoMoE: Breaking the Full-Replica Barrier with a Federation of MoEs
Lorenzo Sani, Zeyu Cao, Meghdad Kurmanji +5
Pre-training Large Language Models (LLMs) typically demands large-scale infrastructure with tightly coupled hardware accelerators. Mixture-of-Experts (MoEs) architectures partially…
PreLort: Prefix-Nested LoRA for Federated Fine-Tuning under Rank Heterogeneity
Muhammad Waseem, Nurbek Tastan, Andrej Jovanovic +4
Federated fine-tuning of large language models using parameter-efficient methods such as LoRA enables privacy-preserving adaptation of foundation models. Heterogeneous hardware res…
Beyond Scaling: Agents Are Heading to the Edge
Chunlin Tian, Dongqi Cai, Wanru Zhao +1
The bottleneck of useful agentic intelligence has shifted from compressing world knowledge into a single model to executing a coordinated system. This position paper argues that pe…
Response-Conditioned Parallel-to-Sequential Orchestration for Multi-Agent Systems
Nurbek Tastan, Alex Iacob, Lorenzo Sani +4
Multi-agent systems can solve complex tasks through collaboration between multiple Large Language Model agents. Existing collaboration frameworks typically operate in either a para…
EdgeFlowerTune: Evaluating Federated LLM Fine-Tuning Under Realistic Edge System Constraints
Jiaxiang Geng, Yiyi Lu, Lunyu Zhao +3
Federated fine-tuning offers a promising paradigm for adapting large language models (LLMs) on edge devices by leveraging the rich, diverse, and continuously generated data from sm…
Task-Centric Personalized Federated Fine-Tuning of Language Models
Gabriel U. Talasso, Meghdad Kurmanji, Allan M. de Souza +2
Federated Learning (FL) has emerged as a promising technique for training language models on distributed and private datasets of diverse tasks. However, aggregating models trained…
-FUM: Federated Unlearning via min--max and -divergence
Radmehr Karimian, Amirhossein Bagheri, Meghdad Kurmanji +2
Federated learning (FL) enables collaborative training while keeping raw data on clients. Deletion requests and the discovery of poisoned data create a need to remove selected cont…
LoRDO: Distributed Low-Rank Optimization with Infrequent Communication
Andrej Jovanović, Alex Iacob, Mher Safaryan +6
Distributed training of foundation models via is limited by interconnect bandwidth. While infrequent communication strategies reduce synchronization frequency, they…
Computational Compliance for AI Regulation: Blueprint for a New Research Domain
Bill Marino, Nicholas D. Lane
The era of AI regulation (AIR) is upon us. But AI systems, we argue, will not be able to comply with these regulations at the necessary speed and scale by continuing to rely on tra…