QUARK: A Framework for Quantum Computing Application Benchmarking

Abstract

Quantum computing (QC) is anticipated to provide a speedup over classical HPC approaches for specific problems in optimization, simulation, and machine learning. With the advances in quantum computing toward practical applications, the need to analyze and compare different quantum solutions increases. While different low-level benchmarks for QC exist, these benchmarks do not provide sufficient insights into real-world application-level performance. We propose an application-centric benchmark method and the QUantum computing Application benchmaRK (QUARK) framework to foster the investigation and creation of application benchmarks for QC. This paper establishes three significant contributions: (1) it makes a case for application-level benchmarks and provides an in-depth "pen and paper" benchmark formulation of two reference problems: robot path and vehicle option optimization from the industrial domain; (2) it proposes the open-source QUARK framework for designing, implementing, executing, and analyzing benchmarks; (3) it provides multiple reference implementations for these two reference problems based on different known, and where needed, extended, classical and quantum algorithmic approaches and analyzes their performance on different types of infrastructures.

Improved version, as submitted to IEEE QCE22 conference

References in corpus (7)

• Supplementary information for "Quantum supremacy using a programmable superconducting processor"
• A Quantum Approximate Optimization Algorithm
• Quantum Phases of Matter on a 256-Atom Programmable Quantum Simulator
• MLPerf Training Benchmark
• An application benchmark for fermionic quantum simulations
• Efficient calculation of gradients in classical simulations of variational quantum algorithms
• A Classically Efficient Quantum Scalable Fermi-Hubbard Benchmark

Cited by in corpus (13)

• MQT Bench: Benchmarking Software and Design Automation Tools for Quantum Computing
• Challenges and Opportunities in Quantum Optimization
• Neutral Atom Quantum Computing Hardware: Performance and End-User Perspective
• Benchmarking quantum computers
• Quantum utility -- definition and assessment of a practical quantum advantage
• Quantum Computing Techniques for Multi-Knapsack Problems
• Recommending Solution Paths for Solving Optimization Problems with Quantum Computing
• Benchmarking Quantum Generative Learning: A Study on Scalability and Noise Resilience using QUARK
• Benchmarking Quantum Computers: Towards a Standard Performance Evaluation Approach
• Application-Oriented Benchmarking of Quantum Generative Learning Using QUARK
• Survey on Computational Applications of Tensor Network Simulations
• Typical Machine Learning Datasets as Low-Depth Quantum Circuits
• Quantum Optimization Benchmarking Library - The Intractable Decathlon