Risk Bounds for Robust Deep Learning

Abstract

It has been observed that certain loss functions can render deep-learning pipelines robust against flaws in the data. In this paper, we support these empirical findings with statistical theory. We especially show that empirical-risk minimization with unbounded, Lipschitz-continuous loss functions, such as the least-absolute deviation loss, Huber loss, Cauchy loss, and Tukey's biweight loss, can provide efficient prediction under minimal assumptions on the data. More generally speaking, our paper provides theoretical evidence for the benefits of robust loss functions in deep learning.

References in corpus (7)

• Ensemble Adversarial Training: Attacks and Defenses
• Towards Deep Learning Models Resistant to Adversarial Attacks
• Adversarial Machine Learning at Scale
• Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey
• Studying Very Low Resolution Recognition Using Deep Networks
• New concentration inequalities for suprema of empirical processes
• Towards Robust Deep Neural Networks

Cited by in corpus (5)

• Activation Functions in Artificial Neural Networks: A Systematic Overview
• Robust Nonparametric Regression with Deep Neural Networks
• Optimization Landscapes of Wide Deep Neural Networks Are Benign
• Targeted Deep Learning: Framework, Methods, and Applications
• Finding the Optimal Dynamic Treatment Regime Using Smooth Fisher Consistent Surrogate Loss