5 papers
AI Benchmark Democratization and Carpentry
Gregor von Laszewski, Wesley Brewer, Jeyan Thiyagalingam +28
Benchmarks are a cornerstone of modern machine learning, enabling reproducibility, comparison, and scientific progress. However, AI benchmarks are increasingly complex, requiring d…
An MLCommons Scientific Benchmarks Ontology
Ben Hawks, Gregor von Laszewski, Matthew D. Sinclair +6
Scientific machine learning research spans diverse domains and data modalities, yet existing benchmark efforts remain siloed and lack standardization. This makes novel and transfor…
Towards Experiment Execution in Support of Community Benchmark Workflows for HPC
Gregor von Laszewski, Wesley Brewer, Sean R. Wilkinson +5
A key hurdle is demonstrating compute resource capability with limited benchmarks. We propose workflow templates as a solution, offering adaptable designs for specific scientific a…
MLCommons Cloud Masking Benchmark with Early Stopping
Varshitha Chennamsetti, Gregor von Laszewski, Ruochen Gu +6
In this paper, we report on work performed for the MLCommons Science Working Group on the cloud masking benchmark. MLCommons is a consortium that develops and maintains several sci…
Whitepaper on Reusable Hybrid and Multi-Cloud Analytics Service Framework
Gregor von Laszewski, Wo Chang, Russell Reinsch +5
Over the last several years, the computation landscape for conducting data analytics has completely changed. While in the past, a lot of the activities have been undertaken in isol…