17 papers
Multi-Agent Systems are Mixtures of Experts: Who Becomes an Influencer?
Franka Bause, Jonas Niederle, Martin Pawelczyk +1
The effectiveness of multi-agent LLM deliberation depends not only on the agents' individual predictions, but also on how they communicate and collaborate. We study this mechanism…
Robustness of Mixtures of Experts to Feature Noise
Dong Sun, Rahul Nittala, Rebekka Burkholz
Despite their practical success, it remains unclear why Mixture of Experts (MoE) models can outperform dense networks beyond sheer parameter scaling. We study an iso-parameter regi…
Fixed Aggregation Features Can Rival GNNs
Celia Rubio-Madrigal, Rebekka Burkholz
Graph neural networks (GNNs) are widely believed to excel at node representation learning through trainable neighborhood aggregations. We challenge this view by introducing Fixed A…
SparseOpt: Addressing Normalization-induced Gradient Skew in Sparse Training
Mohammed Adnan, Rohan Jain, Tom Jacobs +4
Dynamic Sparse Training (DST) methods train neural networks by maintaining sparsity while dynamically adapting the network topology. Despite the promise of reduced computation, DST…
HORST: Composing Optimizer Geometries for Sparse Transformer Training
Tom Jacobs, Rohan Jain, Rebekka Burkholz
Sparsifying transformers remains a fundamental challenge, as standard optimizers fail to simultaneously encourage sparsity and maintain training stability. Effective adaptive optim…
Don't Trust Stubborn Neighbors: A Security Framework for Agentic Networks
Samira Abedini, Sina Mavali, Lea Schönherr +2
Large Language Model (LLM)-based Multi-Agent Systems (MASs) are increasingly deployed for agentic tasks, such as web automation, itinerary planning, and collaborative problem solvi…