518 citations · 1.3k across the 51 of their papers we have counts for
12 papers · 1 filter
DP-HyPO: An Adaptive Private Hyperparameter Optimization Framework
Hua Wang, Sheng Gao, Huanyu Zhang +2
Hyperparameter optimization, also known as hyperparameter tuning, is a widely recognized technique for improving model performance. Regrettably, when training private ML models, ma…
Envisioning Future Deep Learning Theories: Some Basic Concepts and Characteristics
Weijie J. Su
To advance deep learning methodologies in the next decade, a theoretical framework for reasoning about modern neural networks is needed. While efforts are increasing toward demysti…
Oneshot Differentially Private Top-k Selection
Gang Qiao, Weijie J. Su, Li Zhang
Being able to efficiently and accurately select the top- elements with differential privacy is an integral component of various private data analysis tasks. In this paper, we pr…
Weighted Training for Cross-Task Learning
Shuxiao Chen, Koby Crammer, Hangfeng He +2
In this paper, we introduce Target-Aware Weighted Training (TAWT), a weighted training algorithm for cross-task learning based on minimizing a representation-based task distance be…
Exploring Deep Neural Networks via Layer-Peeled Model: Minority Collapse in Imbalanced Training
Cong Fang, Hangfeng He, Qi Long +1
In this paper, we introduce the \textit{Layer-Peeled Model}, a nonconvex yet analytically tractable optimization program, in a quest to better understand deep neural networks that…
Label-Aware Neural Tangent Kernel: Toward Better Generalization and Local Elasticity
Shuxiao Chen, Hangfeng He, Weijie J. Su
As a popular approach to modeling the dynamics of training overparametrized neural networks (NNs), the neural tangent kernels (NTK) are known to fall behind real-world NNs in gener…
Towards Understanding the Dynamics of the First-Order Adversaries
Zhun Deng, Hangfeng He, Jiaoyang Huang +1
An acknowledged weakness of neural networks is their vulnerability to adversarial perturbations to the inputs. To improve the robustness of these models, one of the most popular de…
Toward Better Generalization Bounds with Locally Elastic Stability
Zhun Deng, Hangfeng He, Weijie J. Su
Algorithmic stability is a key characteristic to ensure the generalization ability of a learning algorithm. Among different notions of stability, \emph{uniform stability} is arguab…
On Learning Rates and Schrödinger Operators
Bin Shi, Weijie J. Su, Michael I. Jordan
The learning rate is perhaps the single most important parameter in the training of neural networks and, more broadly, in stochastic (nonconvex) optimization. Accordingly, there ar…
Deep Learning with Gaussian Differential Privacy
Zhiqi Bu, Jinshuo Dong, Qi Long +1
Deep learning models are often trained on datasets that contain sensitive information such as individuals' shopping transactions, personal contacts, and medical records. An increas…
The Local Elasticity of Neural Networks
Hangfeng He, Weijie J. Su
This paper presents a phenomenon in neural networks that we refer to as \textit{local elasticity}. Roughly speaking, a classifier is said to be locally elastic if its prediction at…
Quantifying Intrinsic Uncertainty in Classification via Deep Dirichlet Mixture Networks
Qingyang Wu, He Li, Lexin Li +1
With the widespread success of deep neural networks in science and technology, it is becoming increasingly important to quantify the uncertainty of the predictions produced by deep…