12 citations · 40 across the 13 of their papers we have counts for
14 papers
LLM Safety Alignment is Divergence Estimation in Disguise
Rajdeep Haldar, Ziyi Wang, Qifan Song +2
We present a theoretical framework showing that popular LLM alignment methods, including RLHF and its variants, can be understood as divergence estimators between aligned (safe or…
Adversarial Vulnerability as a Consequence of On-Manifold Inseparibility
Rajdeep Haldar, Yue Xing, Qifan Song +1
Recent works have shown theoretically and empirically that redundant data dimensions are a source of adversarial vulnerability. However, the inverse doesn't seem to hold in practic…
Effect of Ambient-Intrinsic Dimension Gap on Adversarial Vulnerability
Rajdeep Haldar, Yue Xing, Qifan Song
The existence of adversarial attacks on machine learning models imperceptible to a human is still quite a mystery from a theoretical perspective. In this work, we introduce two not…
Theoretical Understanding of In-Context Learning in Shallow Transformers with Unstructured Data
Yue Xing, Xiaofeng Lin, Chenheng Xu +3
Large language models (LLMs) are powerful models that can learn concepts at the inference stage via in-context learning (ICL). While theoretical studies, e.g., \cite{zhang2023train…
Better Representations via Adversarial Training in Pre-Training: A Theoretical Perspective
Yue Xing, Xiaofeng Lin, Qifan Song +3
Pre-training is known to generate universal representations for downstream tasks in large-scale deep learning such as large language models. Existing literature, e.g., \cite{kim202…
Adversarial Training with Generated Data in High-Dimensional Regression: An Asymptotic Study
Yue Xing
In recent years, studies such as \cite{carmon2019unlabeled,gowal2021improving,xing2022artificial} have demonstrated that incorporating additional real or generated data with pseudo…
Benefit of Interpolation in Nearest Neighbor Algorithms
Yue Xing, Qifan Song, Guang Cheng
In some studies \citep[e.g.,][]{zhang2016understanding} of deep learning, it is observed that over-parametrized deep neural networks achieve a small testing error even when the tra…
Unlabeled Data Help: Minimax Analysis and Adversarial Robustness
Yue Xing, Qifan Song, Guang Cheng
The recent proposed self-supervised learning (SSL) approaches successfully demonstrate the great potential of supplementing learning algorithms with additional unlabeled data. Howe…
Adversarially Robust Estimate and Risk Analysis in Linear Regression
Yue Xing, Ruizhi Zhang, Guang Cheng
Adversarially robust learning aims to design algorithms that are robust to small adversarial perturbations on input variables. Beyond the existing studies on the predictive perform…
On the Generalization Properties of Adversarial Training
Yue Xing, Qifan Song, Guang Cheng
Modern machine learning and deep learning models are shown to be vulnerable when testing data are slightly perturbed. Existing theoretical studies of adversarial training algorithm…
Directional Pruning of Deep Neural Networks
Shih-Kang Chao, Zhanyu Wang, Yue Xing +1
In the light of the fact that the stochastic gradient descent (SGD) often finds a flat minimum valley in the training loss, we propose a novel directional pruning method which sear…
Predictive Power of Nearest Neighbors Algorithm under Random Perturbation
Yue Xing, Qifan Song, Guang Cheng
We consider a data corruption scenario in the classical Nearest Neighbors (-NN) algorithm, that is, the testing data are randomly perturbed. Under such a scenario, the impac…
Benefit of Interpolation in Nearest Neighbor Algorithms
Yue Xing, Qifan Song, Guang Cheng
The over-parameterized models attract much attention in the era of data science and deep learning. It is empirically observed that although these models, e.g. deep neural networks,…
Statistical Optimality of Interpolated Nearest Neighbor Algorithms
Yue Xing, Qifan Song, Guang Cheng
In the era of deep learning, understanding over-fitting phenomenon becomes increasingly important. It is observed that carefully designed deep neural networks achieve small testing…