Exploring Robust Architectures for Deep Artificial Neural Networks

Abstract

The architectures of deep artificial neural networks (DANNs) are routinely studied to improve their predictive performance. However, the relationship between the architecture of a DANN and its robustness to noise and adversarial attacks is less explored. We investigate how the robustness of DANNs relates to their underlying graph architectures or structures. This study: (1) starts by exploring the design space of architectures of DANNs using graph-theoretic robustness measures; (2) transforms the graphs to DANN architectures to train/validate/test on various image classification tasks; (3) explores the relationship between the robustness of trained DANNs against noise and adversarial attacks and the robustness of their underlying architectures estimated via graph-theoretic measures. We show that the topological entropy and Olivier-Ricci curvature of the underlying graphs can quantify the robustness performance of DANNs. The said relationship is stronger for complex tasks and large DANNs. Our work will allow autoML and neural architecture search community to explore design spaces of robust and accurate DANNs.

27 pages, 16 figures

References in corpus (5)

• Modeling Realistic Adversarial Attacks against Network Intrusion Detection Systems
• Graph Vulnerability and Robustness: A Survey
• PaSca: a Graph Neural Architecture Search System under the Scalable Paradigm
• POPQORN: Quantifying Robustness of Recurrent Neural Networks
• Fractal and multifractal analysis of complex networks: Estonian network of payments

Cited by in corpus (5)

• Transformers in Time-series Analysis: A Tutorial
• Multimodal Data Integration for Oncology in the Era of Deep Neural Networks: A Review
• EvalAttAI: A Holistic Approach to Evaluating Attribution Maps in Robust and Non-Robust Models
• Task complexity shapes internal representations and robustness in neural networks
• Application of discrete Ricci curvature in pruning randomly wired neural networks: A case study with chest x-ray classification of COVID-19