401 citations · 1.2k across the 88 of their papers we have counts for
27 papers · 1 filter
Nexus: Same Pretraining Loss, Better Downstream Generalization via Common Minima
Huanran Chen, Huaqing Zhang, Xiao Li +3
The foundational capabilities of large language models are acquired during pretraining on internet-scale, highly heterogeneous data mixtures. In this work, we investigate an intere…
Unveiling the Basin-Like Loss Landscape in Large Language Models
Huanran Chen, Yinpeng Dong, Zeming Wei +4
We discover the emergence of \textit{basins} in the loss landscape of large language models. As model scale increases, LLMs become progressively more resilient to random perturbati…
Towards the Worst-case Robustness of Large Language Models
Huanran Chen, Yinpeng Dong, Zeming Wei +2
Recent studies have revealed the vulnerability of large language models to adversarial attacks, where adversaries craft specific input sequences to induce harmful, violent, private…
Scaling Laws for Black box Adversarial Attacks
Chuan Liu, Huanran Chen, Yichi Zhang +2
Adversarial examples exhibit cross-model transferability, enabling threatening black-box attacks on commercial models. Model ensembling, which attacks multiple surrogate models, is…
Efficient Black-box Adversarial Attacks via Bayesian Optimization Guided by a Function Prior
Shuyu Cheng, Yibo Miao, Yinpeng Dong +3
This paper studies the challenging black-box adversarial attack that aims to generate adversarial examples against a black-box model by only using output feedback of the model to i…
Your Diffusion Model is Secretly a Certifiably Robust Classifier
Huanran Chen, Yinpeng Dong, Shitong Shao +4
Generative learning, recognized for its effective modeling of data distributions, offers inherent advantages in handling out-of-distribution instances, especially for enhancing rob…
Learning Sample Difficulty from Pre-trained Models for Reliable Prediction
Peng Cui, Dan Zhang, Zhijie Deng +2
Large-scale pre-trained models have achieved remarkable success in many applications, but how to leverage them to improve the prediction reliability of downstream models is undesir…
GNOT: A General Neural Operator Transformer for Operator Learning
Zhongkai Hao, Zhengyi Wang, Hang Su +6
Learning partial differential equations' (PDEs) solution operators is an essential problem in machine learning. However, there are several challenges for learning operators in prac…
GSmooth: Certified Robustness against Semantic Transformations via Generalized Randomized Smoothing
Zhongkai Hao, Chengyang Ying, Yinpeng Dong +3
Certified defenses such as randomized smoothing have shown promise towards building reliable machine learning systems against -norm bounded attacks. However, existing metho…
A Roadmap for Big Model
Sha Yuan, Hanyu Zhao, Shuai Zhao +97
With the rapid development of deep learning, training Big Models (BMs) for multiple downstream tasks becomes a popular paradigm. Researchers have achieved various outcomes in the c…
Query-Efficient Black-box Adversarial Attacks Guided by a Transfer-based Prior
Yinpeng Dong, Shuyu Cheng, Tianyu Pang +2
Adversarial attacks have been extensively studied in recent years since they can identify the vulnerability of deep learning models before deployed. In this paper, we consider the…
Model-Agnostic Meta-Attack: Towards Reliable Evaluation of Adversarial Robustness
Xiao Yang, Yinpeng Dong, Wenzhao Xiang +3
The vulnerability of deep neural networks to adversarial examples has motivated an increasing number of defense strategies for promoting model robustness. However, the progress is…
Boosting Transferability of Targeted Adversarial Examples via Hierarchical Generative Networks
Xiao Yang, Yinpeng Dong, Tianyu Pang +2
Transfer-based adversarial attacks can evaluate model robustness in the black-box setting. Several methods have demonstrated impressive untargeted transferability, however, it is s…
Accumulative Poisoning Attacks on Real-time Data
Tianyu Pang, Xiao Yang, Yinpeng Dong +2
Collecting training data from untrusted sources exposes machine learning services to poisoning adversaries, who maliciously manipulate training data to degrade the model accuracy.…
Exploring Memorization in Adversarial Training
Yinpeng Dong, Ke Xu, Xiao Yang +4
Deep learning models have a propensity for fitting the entire training set even with random labels, which requires memorization of every training sample. In this paper, we explore…
Automated Decision-based Adversarial Attacks
Qi-An Fu, Yinpeng Dong, Hang Su +1
Deep learning models are vulnerable to adversarial examples, which can fool a target classifier by imposing imperceptible perturbations onto natural examples. In this work, we cons…
Two Coupled Rejection Metrics Can Tell Adversarial Examples Apart
Tianyu Pang, Huishuai Zhang, Di He +5
Correctly classifying adversarial examples is an essential but challenging requirement for safely deploying machine learning models. As reported in RobustBench, even the state-of-t…
Bag of Tricks for Adversarial Training
Tianyu Pang, Xiao Yang, Yinpeng Dong +2
Adversarial training (AT) is one of the most effective strategies for promoting model robustness. However, recent benchmarks show that most of the proposed improvements on AT are l…
Towards Face Encryption by Generating Adversarial Identity Masks
Xiao Yang, Yinpeng Dong, Tianyu Pang +4
As billions of personal data being shared through social media and network, the data privacy and security have drawn an increasing attention. Several attempts have been made to all…
Boosting Adversarial Training with Hypersphere Embedding
Tianyu Pang, Xiao Yang, Yinpeng Dong +3
Adversarial training (AT) is one of the most effective defenses against adversarial attacks for deep learning models. In this work, we advocate incorporating the hypersphere embedd…
Adversarial Distributional Training for Robust Deep Learning
Yinpeng Dong, Zhijie Deng, Tianyu Pang +2
Adversarial training (AT) is among the most effective techniques to improve model robustness by augmenting training data with adversarial examples. However, most existing AT method…
Improving Black-box Adversarial Attacks with a Transfer-based Prior
Shuyu Cheng, Yinpeng Dong, Tianyu Pang +2
We consider the black-box adversarial setting, where the adversary has to generate adversarial perturbations without access to the target models to compute gradients. Previous meth…
Batch Virtual Adversarial Training for Graph Convolutional Networks
Zhijie Deng, Yinpeng Dong, Jun Zhu
We present batch virtual adversarial training (BVAT), a novel regularization method for graph convolutional networks (GCNs). BVAT addresses the shortcoming of GCNs that do not cons…
Towards Interpretable Deep Neural Networks by Leveraging Adversarial Examples
Yinpeng Dong, Fan Bao, Hang Su +1
Sometimes it is not enough for a DNN to produce an outcome. For example, in applications such as healthcare, users need to understand the rationale of the decisions. Therefore, it…