12 papers
The Red Queen Gödel Machine: Co-Evolving Agents and Their Evaluators
Alex Iacob, Andrej JovanoviÄ, William F. Shen +10
Self-improving agents are state-of-the-art (SOTA) on agentic coding benchmarks and have recently been extended to general domains. However, their search methods generally assume a…
FoMoE: Breaking the Full-Replica Barrier with a Federation of MoEs
Lorenzo Sani, Zeyu Cao, Meghdad Kurmanji +5
Pre-training Large Language Models (LLMs) typically demands large-scale infrastructure with tightly coupled hardware accelerators. Mixture-of-Experts (MoEs) architectures partially…
LoRDO: Distributed Low-Rank Optimization with Infrequent Communication
Andrej JovanoviÄ, Alex Iacob, Mher Safaryan +6
Distributed training of foundation models via is limited by interconnect bandwidth. While infrequent communication strategies reduce synchronization frequency, they…
Photon: Federated LLM Pre-Training
Lorenzo Sani, Alex Iacob, Zeyu Cao +8
Scaling large language models (LLMs) demands extensive data and computing resources, which are traditionally constrained to data centers by the high-bandwidth requirements of distr…
Response-Conditioned Parallel-to-Sequential Orchestration for Multi-Agent Systems
Nurbek Tastan, Alex Iacob, Lorenzo Sani +4
Multi-agent systems can solve complex tasks through collaboration between multiple Large Language Model agents. Existing collaboration frameworks typically operate in either a para…
BixBench: a Comprehensive Benchmark for LLM-based Agents in Computational Biology
Ludovico Mitchener, Jon M Laurent, Alex Andonian +6
Large Language Models (LLMs) and LLM-based agents show great promise in accelerating scientific research. Existing benchmarks for measuring this potential and guiding future develo…