Application-Motivated, Holistic Benchmarking of a Full Quantum Computing Stack

Abstract

Quantum computing systems need to be benchmarked in terms of practical tasks they would be expected to do. Here, we propose 3 "application-motivated" circuit classes for benchmarking: deep (relevant for state preparation in the variational quantum eigensolver algorithm), shallow (inspired by IQP-type circuits that might be useful for near-term quantum machine learning), and square (inspired by the quantum volume benchmark). We quantify the performance of a quantum computing system in running circuits from these classes using several figures of merit, all of which require exponential classical computing resources and a polynomial number of classical samples (bitstrings) from the system. We study how performance varies with the compilation strategy used and the device on which the circuit is run. Using systems made available by IBM Quantum, we examine their performance, showing that noise-aware compilation strategies may be beneficial, and that device connectivity and noise levels play a crucial role in the performance of the system according to our benchmarks.

arXiv comment33 pages, 17 figures; v2 - typos corrected; v3 - rearrangements and compression for clarity, reformatted for publication

References in corpus (65)

• Supplementary information for "Quantum supremacy using a programmable superconducting processor"
• A variational eigenvalue solver on a quantum processor
• Hardware-efficient Variational Quantum Eigensolver for Small Molecules and Quantum Magnets
• Supervised learning with quantum enhanced feature spaces
• Quantum computational chemistry
• Characterizing Quantum Supremacy in Near-Term Devices
• Randomized Benchmarking of Quantum Gates
• Expressibility and entangling capability of parameterized quantum circuits for hybrid quantum-classical algorithms
• Quantum Computational Supremacy
• Demonstration of a small programmable quantum computer with atomic qubits
• Robust randomized benchmarking of quantum processes
• Hartree-Fock on a superconducting qubit quantum computer
• Validating quantum computers using randomized model circuits
• Efficient quantum algorithms for simulating sparse Hamiltonians
• Demonstration of the trapped-ion quantum-CCD computer architecture
• Noise tailoring for scalable quantum computation via randomized compiling
• Experimental Comparison of Two Quantum Computing Architectures
• Quantum Approximate Optimization of Non-Planar Graph Problems on a Planar Superconducting Processor
• Characterizing Quantum Gates via Randomized Benchmarking
• A blueprint for demonstrating quantum supremacy with superconducting qubits
• t$|$ket$⟩$ : A Retargetable Compiler for NISQ Devices
• Cloud Quantum Computing of an Atomic Nucleus
• A generative modeling approach for benchmarking and training shallow quantum circuits
• Quantum-assisted quantum compiling
• Quantum algorithms for electronic structure calculations: particle/hole Hamiltonian and optimized wavefunction expansions
• Local random quantum circuits are approximate polynomial-designs
• The Expressive Power of Parameterized Quantum Circuits
• Characterization of addressability by simultaneous randomized benchmarking
• Classical simulation of commuting quantum computations implies collapse of the polynomial hierarchy
• Average-case complexity versus approximate simulation of commuting quantum computations
• Quantum Supremacy and the Complexity of Random Circuit Sampling
• Quantum Approximate Optimization of the Long-Range Ising Model with a Trapped-Ion Quantum Simulator
• Software Mitigation of Crosstalk on Noisy Intermediate-Scale Quantum Computers
• Random Quantum Circuits and Pseudo-Random Operators: Theory and Applications
• Process verification of two-qubit quantum gates by randomized benchmarking
• Effective Hamiltonian models of the cross-resonance gate
• Achieving quantum supremacy with sparse and noisy commuting quantum computations
• Topological and subsystem codes on low-degree graphs with flag qubits
• Instantaneous Quantum Computation
• Verification of quantum computation: An overview of existing approaches
• A flexible high-performance simulator for verifying and benchmarking quantum circuits implemented on real hardware
• The Born Supremacy: Quantum Advantage and Training of an Ising Born Machine
• Three Qubit Randomized Benchmarking
• ScaffCC: Scalable Compilation and Analysis of Quantum Programs
• Performing Quantum Computing Experiments in the Cloud
• What randomized benchmarking actually measures
• Benchmarking the Quantum Approximate Optimization Algorithm
• Establishing the Quantum Supremacy Frontier with a 281 Pflop/s Simulation
• Efficient Discrete Approximations of Quantum Gates
• Direct randomized benchmarking for multi-qubit devices
• A quantum-classical cloud platform optimized for variational hybrid algorithms
• Architectures for quantum simulation showing a quantum speedup
• Convergence Conditions for Random Quantum Circuits
• Phase Gadget Synthesis for Shallow Circuits
• A volumetric framework for quantum computer benchmarks
• Improved success probability with greater circuit depth for the quantum approximate optimization algorithm
• Direct certification of a class of quantum simulations
• On the qubit routing problem
• Generative model benchmarks for superconducting qubits
• Quantum Bootstrapping via Compressed Quantum Hamiltonian Learning
• From randomized benchmarking experiments to gateset circuit fidelity: how to interpret randomized benchmarking decay parameters
• Optimising Clifford Circuits with Quantomatic
• Information Theoretically Secure Hypothesis Test for Temporally Unstructured Quantum Computation (Extended Abstract)
• Information Theoretically Secure Hypothesis Test for Temporally Unstructured Quantum Computation
• Methods for Classically Simulating Noisy Networked Quantum Architectures

Cited by in corpus (19)

• Application-Oriented Performance Benchmarks for Quantum Computing
• Benchmarking quantum computers
• Quantum Volume in Practice: What Users Can Expect from NISQ Devices
• QUARK: A Framework for Quantum Computing Application Benchmarking
• Benchmarking simulated and physical quantum processing units using quantum and hybrid algorithms
• Distributing circuits over heterogeneous, modular quantum computing network architectures
• Quantum utility -- definition and assessment of a practical quantum advantage
• Predicting Good Quantum Circuit Compilation Options
• Interaction graph-based characterization of quantum benchmarks for improving quantum circuit mapping techniques
• Quantum hardware calculations of periodic systems with partition-measurement symmetry verification: simplified models of hydrogen chain and iron crystals
• MQT Predictor: Automatic Device Selection with Device-Specific Circuit Compilation for Quantum Computing
• Quantum algorithms for scientific computing
• Volumetric Benchmarking of Error Mitigation with Qermit
• Benchmarking Quantum Generative Learning: A Study on Scalability and Noise Resilience using QUARK
• Benchmarking Quantum Computers: Towards a Standard Performance Evaluation Approach
• Noise-Adaptive Quantum Compilation Strategies Evaluated with Application-Motivated Benchmarks
• Counting collisions in random circuit sampling for benchmarking quantum computers
• Scalable Full-Stack Benchmarks for Quantum Computers
• End-to-End Quantum Algorithms for the Jones Polynomial