2 papers
cs.LG2026
ResBM: Residual Bottleneck Models for Low-Bandwidth Pipeline Parallelism
Alan Aboudib, Rodrigo Lopez Portillo A., Kalei Brady +1
Unlocking large-scale low-bandwidth decentralized training has the potential to utilize otherwise untapped compute resources. In centralized settings, large-scale multi-node traini…
cs.DC2025
Incentivised Orchestrated Training Architecture (IOTA): A Technical Primer for Release
Felix Quinque, Alan Aboudib, Szymon Fonau +3
In August 2024, Bittensor's Subnet 9 (SN9) demonstrated that a distributed network of incentivized, permissionless actors could each pretrain large language models (LLMs) ranging f…