921 citations · 1.4k across the 54 of their papers we have counts for
32 papers · 1 filter
zkDL: Efficient Zero-Knowledge Proofs of Deep Learning Training
Haochen Sun, Tonghe Bai, Jason Li +1
The recent advancements in deep learning have brought about significant changes in various aspects of people's lives. Meanwhile, these rapid developments have raised concerns about…
Boosting Multitask Learning on Graphs through Higher-Order Task Affinities
Dongyue Li, Haotian Ju, Aneesh Sharma +1
Predicting node labels on a given graph is a widely studied problem with many applications, including community detection and molecular graph prediction. This paper considers predi…
Identification of Negative Transfers in Multitask Learning Using Surrogate Models
Dongyue Li, Huy L. Nguyen, Hongyang R. Zhang
Multitask learning is widely used in practice to train a low-resource target task by augmenting it with multiple related source tasks. Yet, naively combining all the source tasks w…
Generalization in Graph Neural Networks: Improved PAC-Bayesian Bounds on Graph Diffusion
Haotian Ju, Dongyue Li, Aneesh Sharma +1
Graph neural networks are widely used tools for graph prediction tasks. Motivated by their empirical performance, prior works have developed generalization bounds for graph neural…
Understanding the Impact of Adversarial Robustness on Accuracy Disparity
Yuzheng Hu, Fan Wu, Hongyang Zhang +1
While it has long been empirically observed that adversarial robustness may be at odds with standard accuracy and may have further disparate impacts on different classes, it remain…
Direct-Effect Risk Minimization for Domain Generalization
Yuhui Li, Zejia Wu, Chao Zhang +1
We study the problem of out-of-distribution (o.o.d.) generalization where spurious correlations of attributes vary across training and test domains. This is known as the problem of…
A Closer Look at Robustness to L-infinity and Spatial Perturbations and their Composition
Luke Rowe, Benjamin Thérien, Krzysztof Czarnecki +1
In adversarial machine learning, the popular threat model has been the focus of much previous work. While this mathematical definition of imperceptibility successfull…
Building Robust Ensembles via Margin Boosting
Dinghuai Zhang, Hongyang Zhang, Aaron Courville +3
In the context of adversarial robustness, a single model does not usually have enough power to defend against all possible adversarial attacks, and as a result, has sub-optimal rob…
Causal Balancing for Domain Generalization
Xinyi Wang, Michael Saxon, Jiachen Li +3
While machine learning models rapidly advance the state-of-the-art on various real-world tasks, out-of-domain (OOD) generalization remains a challenging problem given the vulnerabi…
Robust Fine-Tuning of Deep Neural Networks with Hessian-based Generalization Guarantees
Haotian Ju, Dongyue Li, Hongyang R. Zhang
We consider fine-tuning a pretrained deep neural network on a target task. We study the generalization properties of fine-tuning to understand the problem of overfitting, which has…
Improved Group Robustness via Classifier Retraining on Independent Splits
Thien Hang Nguyen, Hongyang R. Zhang, Huy Le Nguyen
Deep neural networks trained by minimizing the average risk can achieve strong average performance. Still, their performance for a subgroup may degrade if the subgroup is underrepr…
Correct-N-Contrast: A Contrastive Approach for Improving Robustness to Spurious Correlations
Michael Zhang, Nimit S. Sohoni, Hongyang R. Zhang +2
Spurious correlations pose a major challenge for robust machine learning. Models trained with empirical risk minimization (ERM) may learn to rely on correlations between class labe…
Boosting Barely Robust Learners: A New Perspective on Adversarial Robustness
Avrim Blum, Omar Montasser, Greg Shakhnarovich +1
We present an oracle-efficient algorithm for boosting the adversarial robustness of barely robust learners. Barely robust learning algorithms learn predictors that are adversariall…
A Law of Robustness beyond Isoperimetry
Yihan Wu, Heng Huang, Hongyang Zhang
We study the robust interpolation problem of arbitrary data distributions supported on a bounded space and propose a two-fold law of robustness. Robust interpolation refers to the…
Improved Regularization and Robustness for Fine-tuning in Neural Networks
Dongyue Li, Hongyang R. Zhang
A widely used algorithm for transfer learning is fine-tuning, where a pre-trained model is fine-tuned on a target task with a small amount of labeled data. When the capacity of the…
Self-Adaptive Training: Bridging Supervised and Self-Supervised Learning
Lang Huang, Chao Zhang, Hongyang Zhang
We propose self-adaptive training -- a unified training algorithm that dynamically calibrates and enhances training processes by model predictions without incurring an extra comput…
An Analysis of Robustness of Non-Lipschitz Networks
Maria-Florina Balcan, Avrim Blum, Dravyansh Sharma +1
Despite significant advances, deep networks remain highly susceptible to adversarial attack. One fundamental challenge is that small input perturbations can often produce large mov…
Adversarial Robustness of Stabilized NeuralODEs Might be from Obfuscated Gradients
Yifei Huang, Yaodong Yu, Hongyang Zhang +2
In this paper we introduce a provably stable architecture for Neural Ordinary Differential Equations (ODEs) which achieves non-trivial adversarial robustness under white-box advers…
Learning Over-Parametrized Two-Layer ReLU Neural Networks beyond NTK
Yuanzhi Li, Tengyu Ma, Hongyang R. Zhang
We consider the dynamic of gradient descent for learning a two-layer neural network. We assume the input is drawn from a Gaussian distribution and the label of $…
Understanding and Improving Information Transfer in Multi-Task Learning
Sen Wu, Hongyang R. Zhang, Christopher Ré
We investigate multi-task learning approaches that use a shared feature representation for all tasks. To better understand the transfer of task information, we study an architectur…
On the Generalization Effects of Linear Transformations in Data Augmentation
Sen Wu, Hongyang R. Zhang, Gregory Valiant +1
Data augmentation is a powerful technique to improve performance in applications such as image and text classification tasks. Yet, there is little rigorous understanding of why and…
Random Smoothing Might be Unable to Certify Robustness for High-Dimensional Images
Avrim Blum, Travis Dick, Naren Manoj +1
We show a hardness result for random smoothing to achieve certified adversarial robustness against attacks in the ball of radius when . Although random smoothing…
A Closer Look at Accuracy vs. Robustness
Yao-Yuan Yang, Cyrus Rashtchian, Hongyang Zhang +2
Current methods for training robust networks lead to a drop in test accuracy, which has led prior works to posit that a robustness-accuracy tradeoff may be inevitable in deep learn…
Self-Adaptive Training: beyond Empirical Risk Minimization
Lang Huang, Chao Zhang, Hongyang Zhang
We propose self-adaptive training---a new training algorithm that dynamically corrects problematic training labels by model predictions without incurring extra computational cost--…