3 papers
cs.DC2026
A Tabular Schedule Abstraction for Communication-Aware Evaluation of Pipeline-Parallel LLM Training
Daniel Barley, Jonathan Leis, Benjamin Klenk +1
Pipeline parallelism is a key technique for distributed training of large language models because it reduces per-device parameter and activation memory. However, comparing pipeline…
cs.AR2025
Scalable and Efficient Intra- and Inter-node Interconnection Networks for Post-Exascale Supercomputers and Data centers
Joaquin Tarraga-Moreno, Daniel Barley, Francisco J. Andujar Munoz +5
The rapid growth of data-intensive applications such as generative AI, scientific simulations, and large-scale analytics is driving modern supercomputers and data centers toward in…
cs.LG2024
Less Memory Means smaller GPUs: Backpropagation with Compressed Activations
Daniel Barley, Holger Fröning
The ever-growing scale of deep neural networks (DNNs) has lead to an equally rapid growth in computational resource requirements. Many recent architectures, most prominently Large…