20 citations · 36 across the 18 of their papers we have counts for
3 papers · 1 filter
A Tabular Schedule Abstraction for Communication-Aware Evaluation of Pipeline-Parallel LLM Training
Daniel Barley, Jonathan Leis, Benjamin Klenk +1
Pipeline parallelism is a key technique for distributed training of large language models because it reduces per-device parameter and activation memory. However, comparing pipeline…
On Performance Analysis of Graphcore IPUs: Analyzing Squared and Skewed Matrix Multiplication
S. -Kazem Shekofteh, Christian Alles, Nils Kochendörfer +1
In recent decades, High Performance Computing (HPC) has undergone significant enhancements, particularly in the realm of hardware platforms, aimed at delivering increased processin…
Reducing Memory Requirements for the IPU using Butterfly Factorizations
S. -Kazem Shekofteh, Christian Alles, Holger Fröning
High Performance Computing (HPC) benefits from different improvements during last decades, specially in terms of hardware platforms to provide more processing power while maintaini…