921 citations · 1.4k across the 54 of their papers we have counts for
54 papers
RAIN: Your Language Models Can Align Themselves without Finetuning
Yuhui Li, Fangyun Wei, Jinjing Zhao +2
Large language models (LLMs) often demonstrate inconsistencies with human preferences. Previous research typically gathered human preference data and then aligned the pre-trained m…
zkDL: Efficient Zero-Knowledge Proofs of Deep Learning Training
Haochen Sun, Tonghe Bai, Jason Li +1
The recent advancements in deep learning have brought about significant changes in various aspects of people's lives. Meanwhile, these rapid developments have raised concerns about…
Boosting Multitask Learning on Graphs through Higher-Order Task Affinities
Dongyue Li, Haotian Ju, Aneesh Sharma +1
Predicting node labels on a given graph is a widely studied problem with many applications, including community detection and molecular graph prediction. This paper considers predi…
Identification of Negative Transfers in Multitask Learning Using Surrogate Models
Dongyue Li, Huy L. Nguyen, Hongyang R. Zhang
Multitask learning is widely used in practice to train a low-resource target task by augmenting it with multiple related source tasks. Yet, naively combining all the source tasks w…
Generalization in Graph Neural Networks: Improved PAC-Bayesian Bounds on Graph Diffusion
Haotian Ju, Dongyue Li, Aneesh Sharma +1
Graph neural networks are widely used tools for graph prediction tasks. Motivated by their empirical performance, prior works have developed generalization bounds for graph neural…
Towards Robust Dataset Learning
Yihan Wu, Xinda Li, Florian Kerschbaum +2
Adversarial training has been actively studied in recent computer vision research to improve the robustness of models. However, due to the huge computational cost of generating adv…
Understanding the Impact of Adversarial Robustness on Accuracy Disparity
Yuzheng Hu, Fan Wu, Hongyang Zhang +1
While it has long been empirically observed that adversarial robustness may be at odds with standard accuracy and may have further disparate impacts on different classes, it remain…
Direct-Effect Risk Minimization for Domain Generalization
Yuhui Li, Zejia Wu, Chao Zhang +1
We study the problem of out-of-distribution (o.o.d.) generalization where spurious correlations of attributes vary across training and test domains. This is known as the problem of…
A Closer Look at Robustness to L-infinity and Spatial Perturbations and their Composition
Luke Rowe, Benjamin Thérien, Krzysztof Czarnecki +1
In adversarial machine learning, the popular threat model has been the focus of much previous work. While this mathematical definition of imperceptibility successfull…
Recovery from Non-Decomposable Distance Oracles
Zhuangfei Hu, Xinda Li, David P. Woodruff +2
A line of work has looked at the problem of recovering an input from distance queries. In this setting, there is an unknown sequence , and one chooses a set…
RetrievalGuard: Provably Robust 1-Nearest Neighbor Image Retrieval
Yihan Wu, Hongyang Zhang, Heng Huang
Recent research works have shown that image retrieval models are vulnerable to adversarial attacks, where slightly modified test inputs could lead to problematic retrieval results.…
Building Robust Ensembles via Margin Boosting
Dinghuai Zhang, Hongyang Zhang, Aaron Courville +3
In the context of adversarial robustness, a single model does not usually have enough power to defend against all possible adversarial attacks, and as a result, has sub-optimal rob…
Causal Balancing for Domain Generalization
Xinyi Wang, Michael Saxon, Jiachen Li +3
While machine learning models rapidly advance the state-of-the-art on various real-world tasks, out-of-domain (OOD) generalization remains a challenging problem given the vulnerabi…
Robust Fine-Tuning of Deep Neural Networks with Hessian-based Generalization Guarantees
Haotian Ju, Dongyue Li, Hongyang R. Zhang
We consider fine-tuning a pretrained deep neural network on a target task. We study the generalization properties of fine-tuning to understand the problem of overfitting, which has…
Certified Error Control of Candidate Set Pruning for Two-Stage Relevance Ranking
Minghan Li, Xinyu Zhang, Ji Xin +2
In information retrieval (IR), candidate set pruning has been commonly used to speed up two-stage relevance ranking. However, such an approach lacks accurate error control and ofte…
Improved Group Robustness via Classifier Retraining on Independent Splits
Thien Hang Nguyen, Hongyang R. Zhang, Huy Le Nguyen
Deep neural networks trained by minimizing the average risk can achieve strong average performance. Still, their performance for a subgroup may degrade if the subgroup is underrepr…
Correct-N-Contrast: A Contrastive Approach for Improving Robustness to Spurious Correlations
Michael Zhang, Nimit S. Sohoni, Hongyang R. Zhang +2
Spurious correlations pose a major challenge for robust machine learning. Models trained with empirical risk minimization (ERM) may learn to rely on correlations between class labe…
Boosting Barely Robust Learners: A New Perspective on Adversarial Robustness
Avrim Blum, Omar Montasser, Greg Shakhnarovich +1
We present an oracle-efficient algorithm for boosting the adversarial robustness of barely robust learners. Barely robust learning algorithms learn predictors that are adversariall…
A Law of Robustness beyond Isoperimetry
Yihan Wu, Heng Huang, Hongyang Zhang
We study the robust interpolation problem of arbitrary data distributions supported on a bounded space and propose a two-fold law of robustness. Robust interpolation refers to the…
Towards Transferable Unrestricted Adversarial Examples with Minimum Changes
Fangcheng Liu, Chao Zhang, Hongyang Zhang
Transfer-based adversarial example is one of the most important classes of black-box attacks. However, there is a trade-off between transferability and imperceptibility of the adve…
Improved Regularization and Robustness for Fine-tuning in Neural Networks
Dongyue Li, Hongyang R. Zhang
A widely used algorithm for transfer learning is fine-tuning, where a pre-trained model is fine-tuned on a target task with a small amount of labeled data. When the capacity of the…
Unrestricted Adversarial Attacks on ImageNet Competition
Yuefeng Chen, Xiaofeng Mao, Yuan He +34
Many works have investigated the adversarial attacks or defenses under the settings where a bounded and imperceptible perturbation can be added to the input. However in the real-wo…
Self-Adaptive Training: Bridging Supervised and Self-Supervised Learning
Lang Huang, Chao Zhang, Hongyang Zhang
We propose self-adaptive training -- a unified training algorithm that dynamically calibrates and enhances training processes by model predictions without incurring an extra comput…
Precise High-Dimensional Asymptotics for Quantifying Heterogeneous Transfers
Fan Yang, Hongyang R. Zhang, Sen Wu +2
The problem of learning one task using samples from another task is central to transfer learning. In this paper, we focus on answering the following question: when does combining t…