The Second International Verification of Neural Networks Competition (VNN-COMP 2021): Summary and Results

Abstract

This report summarizes the second International Verification of Neural Networks Competition (VNN-COMP 2021), held as a part of the 4th Workshop on Formal Methods for ML-Enabled Autonomous Systems that was collocated with the 33rd International Conference on Computer-Aided Verification (CAV). Twelve teams participated in this competition. The goal of the competition is to provide an objective comparison of the state-of-the-art methods in neural network verification, in terms of scalability and speed. Along this line, we used standard formats (ONNX for neural networks and VNNLIB for specifications), standard hardware (all tools are run by the organizers on AWS), and tool parameters provided by the tool authors. This report summarizes the rules, benchmarks, participating tools, results, and lessons learned from this competition.

References in corpus (3)

• Improved Branch and Bound for Neural Network Verification via Lagrangian Decomposition
• Scaling the Convex Barrier with Sparse Dual Algorithms
• Neural Network Branch-and-Bound for Neural Network Verification

Cited by in corpus (7)

• First Three Years of the International Verification of Neural Networks Competition (VNN-COMP)
• Verifying Controllers with Convolutional Neural Network-based Perception: A Case for Intelligible, Safe, and Precise Abstractions
• Neural Network Compression of ACAS Xu Early Prototype is Unsafe: Closed-Loop Verification through Quantized State Backreachability
• The Black-Box Simplex Architecture for Runtime Assurance of Autonomous CPS
• The Power of Typed Affine Decision Structures: A Case Study
• Training Certifiably Robust Neural Networks with Efficient Local Lipschitz Bounds
• Reachability Analysis of Neural Networks with Uncertain Parameters