10 citations · 12 across the 4 of their papers we have counts for
5 papers
Decoupled DiLoCo for Resilient Distributed Pre-training
Arthur Douillard, Keith Rush, Yani Donchev +14
Modern large-scale language model pre-training relies heavily on the single program multiple data (SPMD) paradigm, which requires tight coupling across accelerators. Due to this co…
Generating QM1B with PySCF
Alexander Mathiasen, Hatem Helal, Kerstin Klaser +6
The emergence of foundation models in Computer Vision and Natural Language Processing have resulted in immense progress on downstream tasks. This progress was enabled by datasets w…
Towards Foundational Models for Molecular Learning on Large-Scale Multi-Task Datasets
Dominique Beaini, Shenyang Huang, Joao Alex Cunha +32
Recently, pre-trained foundation models have enabled significant advancements in multiple fields. In molecular machine learning, however, where datasets are often hand-curated, and…
GPS++: Reviving the Art of Message Passing for Molecular Property Prediction
Dominic Masters, Josef Dean, Kerstin Klaser +9
We present GPS++, a hybrid Message Passing Neural Network / Graph Transformer model for molecular property prediction. Our model integrates a well-tuned local message passing compo…
GPS++: An Optimised Hybrid MPNN/Transformer for Molecular Property Prediction
Dominic Masters, Josef Dean, Kerstin Klaser +7
This technical report presents GPS++, the first-place solution to the Open Graph Benchmark Large-Scale Challenge (OGB-LSC 2022) for the PCQM4Mv2 molecular property prediction task.…