9 citations · 19 across the 7 of their papers we have counts for
7 papers
Myths and Legends in High-Performance Computing
Satoshi Matsuoka, Jens Domke, Mohamed Wahib +2
In this thought-provoking article, we discuss certain myths and legends that are folklore among members of the high-performance computing community. We gathered these myths from co…
Outliers Dimensions that Disrupt Transformers Are Driven by Frequency
Giovanni Puccetti, Anna Rogers, Aleksandr Drozd +1
While Transformer-based language models are generally very robust to pruning, there is the recently discovered outlier phenomenon: disabling only 48 out of 110M parameters in BERT-…
Preparing for the Future -- Rethinking Proxy Apps
Satoshi Matsuoka, Jens Domke, Mohamed Wahib +5
A considerable amount of research and engineering went into designing proxy applications, which represent common high-performance computing workloads, to co-design and evaluate the…
MLPerf HPC: A Holistic Benchmark Suite for Scientific Machine Learning on HPC Systems
Steven Farrell, Murali Emani, Jacob Balma +40
Scientific communities are increasingly adopting machine learning and deep learning models in their applications to accelerate scientific insights. High performance computing syste…
Generalization in NLI: Ways (Not) To Go Beyond Simple Heuristics
Prajjwal Bhargava, Aleksandr Drozd, Anna Rogers
Much of recent progress in NLU was shown to be due to models' learning dataset-specific heuristics. We conduct a case study of generalization in NLI (from MNLI to the adversarially…
Matrix Engines for High Performance Computing:A Paragon of Performance or Grasping at Straws?
Jens Domke, Emil Vatai, Aleksandr Drozd +8
Matrix engines or units, in different forms and affinities, are becoming a reality in modern processors; CPUs and otherwise. The current and dominant algorithmic approach to Deep L…
Scaling Distributed Deep Learning Workloads beyond the Memory Capacity with KARMA
Mohamed Wahib, Haoyu Zhang, Truong Thao Nguyen +5
The dedicated memory of hardware accelerators can be insufficient to store all weights and/or intermediate states of large deep learning models. Although model parallelism is a via…