22 citations · 34 across the 6 of their papers we have counts for
6 papers
A critical look at the evaluation of GNNs under heterophily: Are we really making progress?
Oleg Platonov, Denis Kuznedelev, Michael Diskin +2
Node classification is a classical graph machine learning task on which Graph Neural Networks (GNNs) have recently achieved strong results. However, it is often believed that stand…
SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficient
Max Ryabinin, Tim Dettmers, Michael Diskin +1
Many deep learning applications benefit from using large models with billions of parameters. Training these models is notoriously expensive due to the need for specialized HPC clus…
Training Transformers Together
Alexander Borzunov, Max Ryabinin, Tim Dettmers +5
The infrastructure necessary for training state-of-the-art models is becoming overly expensive, which makes training such models affordable only to large corporations and instituti…
Distributed Methods with Compressed Communication for Solving Variational Inequalities, with Theoretical Guarantees
Aleksandr Beznosikov, Peter Richtárik, Michael Diskin +2
Variational inequalities in general and saddle point problems in particular are increasingly relevant in machine learning applications, including adversarial learning, GANs, transp…
Secure Distributed Training at Scale
Eduard Gorbunov, Alexander Borzunov, Michael Diskin +1
Many areas of deep learning benefit from using increasingly larger neural networks trained on public data, as is the case for pre-trained models for NLP and computer vision. Traini…
Distributed Deep Learning in Open Collaborations
Michael Diskin, Alexey Bukhtiyarov, Max Ryabinin +13
Modern deep learning applications require increasingly more compute to train state-of-the-art models. To address this demand, large corporations and institutions use dedicated High…