Benchmarking Quantum Computers: Towards a Standard Performance Evaluation Approach
arXiv:2407.10941 · doi:10.1088/2058-9565/ae4269
Abstract
The technological development of increasingly larger quantum processors on different quantum platforms raises the problem of how to fairly compare their performance, known as quantum benchmarking of quantum processors. This is a challenge that computer scientists have already faced when comparing classical processors, leading to the development of various mathematical tools to address it, but also to the identification of the limits of this problem. In this work, we briefly review the most important aspects of both classical processor benchmarks and the metrics comprising them, providing precise definitions and analyzing the quality attributes that they should exhibit. Subsequently, we analyze the intrinsic properties that characterize the paradigm of quantum computing and hinder the naive transfer of strategies from classical benchmarking. However, we can still leverage some of the lessons learned such as the quality attributes of a \textit{good} benchmark. Additionally, we review some of the most important metrics and benchmarks for quantum processors proposed in the literature, assessing what quality attributes they fulfill. Finally, we propose general guidelines for quantum benchmarking. These guidelines aim to pave the way for establishing a roadmap towards standardizing the performance evaluation of quantum devices, ultimately leading to the creation of an organization akin to the Standard Performance Evaluation Corporation (SPEC).
24 pages, 4 figures
References in corpus (61)
- Quantum Computing in the NISQ era and beyond
- Supplementary information for "Quantum supremacy using a programmable superconducting processor"
- A variational eigenvalue solver on a quantum processor
- Surface codes: Towards practical large-scale quantum computation
- Probing many-body dynamics on a 51-atom quantum simulator
- Quantum computational advantage using photons
- Quantum Error Correction for Quantum Memories
- Characterizing Quantum Supremacy in Near-Term Devices
- Predicting Many Properties of a Quantum System from Very Few Measurements
- Strong quantum computational advantage using a superconducting quantum processor
- Suppressing quantum errors by scaling a surface code logical qubit
- Observation of a Many-Body Dynamical Phase Transition with a 53-Qubit Quantum Simulator
- Quantum algorithms: an overview
- Quantum Phases of Matter on a 256-Atom Programmable Quantum Simulator
- Quantum random access memory
- Scalable Quantum Simulation of Molecular Energies
- Demonstration of a small programmable quantum computer with atomic qubits
- Robust randomized benchmarking of quantum processes
- Validating quantum computers using randomized model circuits
- Exact and Approximate Unitary 2-Designs: Constructions and Applications
- Scalable Noise Estimation with Random Unitary Operators
- A blueprint for demonstrating quantum supremacy with superconducting qubits
- Quantum chemistry calculations on a trapped-ion quantum simulator
- Phase-Programmable Gaussian Boson Sampling Using Stimulated Squeezed Light
- Efficient measurement of quantum gate error by interleaved randomized benchmarking
- A tunable coupling scheme for implementing high-fidelity two-qubit gates
- Repeated Quantum Error Detection in a Surface Code
- Quantum certification and benchmarking
- Experimental investigation of performance differences between Coherent Ising Machines and a quantum annealer
- NetSquid, a NETwork Simulator for QUantum Information using Discrete events
- Estimating the Coherence of Noise
- Leakage reduction in fast superconducting qubit gates via optimal control
- Verification of quantum computation: An overview of existing approaches
- Measuring the Capabilities of Quantum Computers
- Overview and Comparison of Gate Level Quantum Software Platforms
- SeQUeNCe: A Customizable Discrete-Event Simulator of Quantum Networks
- A volumetric framework for quantum computer benchmarks
- Scalable randomized benchmarking of quantum computers using mirror circuits
- On Testing Quantum Programs
- Impact of qubit connectivity on quantum algorithm performance
- Enhancing Dispersive Readout of Superconducting Qubits Through Dynamic Control of the Dispersive Shift: Experiment and Theory
- QUARK: A Framework for Quantum Computing Application Benchmarking
- Benchmarking quantum co-processors in an application-centric, hardware-agnostic and scalable way
- Machine learning assisted readout of trapped-ion qubits
- Deterministic event-based simulation of quantum interference
- Application-Motivated, Holistic Benchmarking of a Full Quantum Computing Stack
- Quantum Analogue Computing
- Formal Verification of Quantum Programs: Theory, Tools and Challenges
- Random circuit block-encoded matrix and a proposal of quantum LINPACK benchmark
- Performance of a Quantum Annealer for Ising Ground State Computations on Chimera Graphs
- Characterizing the Reproducibility of Noisy Quantum Circuits
- Quantum advantage for computations with limited space
- Noisy intermediate-scale quantum simulation of the one-dimensional wave equation
- Benchmarking digital quantum simulations above hundreds of qubits using quantum critical dynamics
- Benchmarking quantum gates and circuits
- Fully scalable randomized benchmarking without motion reversal
- Enhanced Qubit Readout via Reinforcement Learning
- Learning a quantum channel from its steady-state
- Evaluating Three Levels of Quantum Metrics on Quantum-Inspire Hardware
- Discrete-event simulation of quantum walks
- Counting collisions in random circuit sampling for benchmarking quantum computers