157 citations · 163 across the 6 of their papers we have counts for
6 papers
Zorse: Optimizing LLM Training Efficiency on Heterogeneous GPU Clusters
Runsheng Benson Guo, Utkarsh Anand, Khuzaima Daudjee +1
Large language models (LLMs) require vast amounts of GPU compute to train, but limited availability and high costs of GPUs make homogeneous clusters impractical for many organizati…
Cephalo: Harnessing Heterogeneous GPU Clusters for Training Transformer Models
Runsheng Benson Guo, Utkarsh Anand, Arthur Chen +1
Training transformer models requires substantial GPU compute and memory resources. In homogeneous clusters, distributed strategies allocate resources evenly, but this approach is i…
FreeRide: Harvesting Bubbles in Pipeline Parallelism
Jiashu Zhang, Zihan Pan, Molly +3
The occurrence of bubbles in pipeline parallelism is an inherent limitation that can account for more than 40% of the large language model (LLM) training time and is one of the mai…
The Future is Big Graphs! A Community View on Graph Processing Systems
Sherif Sakr, Angela Bonifati, Hannes Voigt +38
Graphs are by nature unifying abstractions that can leverage interconnectedness to represent, explore, predict, and explain real- and digital-world phenomena. Although real users a…
Clustering RDF Databases Using Tunable-LSH
Güneş Aluç, M. Tamer Özsu, Khuzaima Daudjee
The Resource Description Framework (RDF) is a W3C standard for representing graph-structured data, and SPARQL is the standard query language for RDF. Recent advances in Information…
An Easy-to-use Scalable Framework for Parallel Recursive Backtracking
Faisal N. Abu-Khzam, Khuzaima Daudjee, Amer E. Mouawad +1
Supercomputers are equipped with an increasingly large number of cores to use computational power as a way of solving problems that are otherwise intractable. Unfortunately, gettin…