518 citations · 1.3k across the 51 of their papers we have counts for
53 papers
High-Dimensional Statistics: Reflections on Progress and Open Problems
Arian Maleki, Subhabrata Sen, Sivaraman Balakrishnan +9
Over the past two decades, the field of high-dimensional statistics has experienced substantial progress, driven largely by technological advances that have dramatically reduced th…
DP-HyPO: An Adaptive Private Hyperparameter Optimization Framework
Hua Wang, Sheng Gao, Huanyu Zhang +2
Hyperparameter optimization, also known as hyperparameter tuning, is a widely recognized technique for improving model performance. Regrettably, when training private ML models, ma…
The Implicit Regularization of Dynamical Stability in Stochastic Gradient Descent
Lei Wu, Weijie J. Su
In this paper, we study the implicit regularization of stochastic gradient descent (SGD) through the lens of {\em dynamical stability} (Wu et al., 2018). We start by revising exist…
Isotonic Mechanism for Exponential Family Estimation in Machine Learning Peer Review
Yuling Yan, Weijie J. Su, Jianqing Fan
In 2023, the International Conference on Machine Learning (ICML) required authors with multiple submissions to rank their submissions based on perceived quality. In this paper, we…
Advancing Differential Privacy: Where We Are Now and Future Directions for Real-World Deployment
Rachel Cummings, Damien Desfontaines, David Evans +21
In this article, we present a detailed review of current practices and state-of-the-art methodologies in the field of differential privacy (DP), with a focus of advancing DP's depl…
On Quantum Speedups for Nonconvex Optimization via Quantum Tunneling Walks
Yizhou Liu, Weijie J. Su, Tongyang Li
Classical algorithms are often not effective for solving nonconvex optimization problems where local minima are separated by high barriers. In this paper, we explore possible quant…
The alignment property of SGD noise and how it helps select flat minima: A stability analysis
Lei Wu, Mingze Wang, Weijie Su
The phenomenon that stochastic gradient descent (SGD) favors flat minima has played a critical role in understanding the implicit regularization of SGD. In this paper, we provide a…
Edgeworth Accountant: An Analytical Approach to Differential Privacy Composition
Hua Wang, Sheng Gao, Huanyu Zhang +3
In privacy-preserving data analysis, many procedures and algorithms are structured as compositions of multiple private building blocks. As such, an important question is how to eff…
Envisioning Future Deep Learning Theories: Some Basic Concepts and Characteristics
Weijie J. Su
To advance deep learning methodologies in the next decade, a theoretical framework for reasoning about modern neural networks is needed. While efforts are increasing toward demysti…
Oneshot Differentially Private Top-k Selection
Gang Qiao, Weijie J. Su, Li Zhang
Being able to efficiently and accurately select the top- elements with differential privacy is an integral component of various private data analysis tasks. In this paper, we pr…
Weighted Training for Cross-Task Learning
Shuxiao Chen, Koby Crammer, Hangfeng He +2
In this paper, we introduce Target-Aware Weighted Training (TAWT), a weighted training algorithm for cross-task learning based on minimizing a representation-based task distance be…
Characterizing the SLOPE Trade-off: A Variational Perspective and the Donoho-Tanner Limit
Zhiqi Bu, Jason Klusowski, Cynthia Rush +1
Sorted l1 regularization has been incorporated into many methods for solving high-dimensional statistical estimation problems, including the SLOPE estimator in linear regression. I…
Rejoinder: Gaussian Differential Privacy
Jinshuo Dong, Aaron Roth, Weijie J. Su
In this rejoinder, we aim to address two broad issues that cover most comments made in the discussion. First, we discuss some theoretical aspects of our work and comment on how thi…
A Central Limit Theorem for Differentially Private Query Answering
Jinshuo Dong, Weijie J. Su, Linjun Zhang
Perhaps the single most important use case for differential privacy is to privately answer numerical queries, which is usually achieved by adding noise to the answer vector. The ce…
Minimax Estimation for Personalized Federated Learning: An Alternative between FedAvg and Local Training?
Shuxiao Chen, Qinqing Zheng, Qi Long +1
A widely recognized difficulty in federated learning arises from the statistical heterogeneity among clients: local datasets often originate from distinct yet not entirely unrelate…
Federated -Differential Privacy
Qinqing Zheng, Shuxiao Chen, Qi Long +1
Federated learning (FL) is a training paradigm where the clients collaboratively learn models by repeatedly sharing information without compromising much on the privacy of their lo…
Exploring Deep Neural Networks via Layer-Peeled Model: Minority Collapse in Imbalanced Training
Cong Fang, Hangfeng He, Qi Long +1
In this paper, we introduce the \textit{Layer-Peeled Model}, a nonconvex yet analytically tractable optimization program, in a quest to better understand deep neural networks that…
Label-Aware Neural Tangent Kernel: Toward Better Generalization and Local Elasticity
Shuxiao Chen, Hangfeng He, Weijie J. Su
As a popular approach to modeling the dynamics of training overparametrized neural networks (NNs), the neural tangent kernels (NTK) are known to fall behind real-world NNs in gener…
Towards Understanding the Dynamics of the First-Order Adversaries
Zhun Deng, Hangfeng He, Jiaoyang Huang +1
An acknowledged weakness of neural networks is their vulnerability to adversarial perturbations to the inputs. To improve the robustness of these models, one of the most popular de…
Toward Better Generalization Bounds with Locally Elastic Stability
Zhun Deng, Hangfeng He, Weijie J. Su
Algorithmic stability is a key characteristic to ensure the generalization ability of a learning algorithm. Among different notions of stability, \emph{uniform stability} is arguab…
Precise High-Dimensional Asymptotics for Quantifying Heterogeneous Transfers
Fan Yang, Hongyang R. Zhang, Sen Wu +2
The problem of learning one task using samples from another task is central to transfer learning. In this paper, we focus on answering the following question: when does combining t…
Benign Overfitting and Noisy Features
Zhu Li, Weijie Su, Dino Sejdinovic
Modern machine learning often operates in the regime where the number of parameters is much higher than the number of data points, with zero training loss and yet good generalizati…
A Power Analysis for Model-X Knockoffs with -Regularized Statistics
Asaf Weinstein, Weijie J. Su, Małgorzata Bogdan +2
Variable selection properties of procedures utilizing penalized-likelihood estimates is a central topic in the study of high dimensional linear regression problems. Existing litera…
The Complete Lasso Tradeoff Diagram
Hua Wang, Yachong Yang, Zhiqi Bu +1
A fundamental problem in the high-dimensional regression is to understand the tradeoff between type I and type II errors or, equivalently, false discovery rate (FDR) and power in v…