3DB: A Framework for Debugging Computer Vision Models
arXiv:2106.03805
Abstract
We introduce 3DB: an extendable, unified framework for testing and debugging vision models using photorealistic simulation. We demonstrate, through a wide range of use cases, that 3DB allows users to discover vulnerabilities in computer vision systems and gain insights into how models make decisions. 3DB captures and generalizes many robustness analyses from prior work, and enables one to study their interplay. Finally, we find that the insights generated by the system transfer to the physical world. We are releasing 3DB as a library (https://github.com/3db/3db) alongside a set of example analyses, guides, and documentation: https://3db.github.io/3db/ .
References in corpus (20)
- Deep Residual Learning for Image Recognition
- Axiomatic Attribution for Deep Networks
- ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness
- Sanity Checks for Saliency Maps
- Unity: A General Platform for Intelligent Agents
- AI2-THOR: An Interactive 3D Environment for Visual AI
- CARLA: An Open Urban Driving Simulator
- Exploring the Landscape of Spatial Robustness
- Building Generalizable Agents with a Realistic and Rich 3D Environment
- Natural Adversarial Examples
- robosuite: A Modular Simulation Framework and Benchmark for Robot Learning
- DPatch: An Adversarial Patch Attack on Object Detectors
- Adversarial Examples Are a Natural Consequence of Test Error in Noise
- Benchmarking Neural Network Robustness to Common Corruptions and Surface Variations
- Physical Adversarial Examples for Object Detectors
- Testing Robustness Against Unforeseen Adversaries
- Noise or Signal: The Role of Image Backgrounds in Object Recognition
- Adversarial Examples for Semantic Image Segmentation
- Towards Analyzing Semantic Robustness of Deep Neural Networks
- Analyzing and Improving Neural Networks by Generating Semantic Counterexamples through Differentiable Rendering