124 citations · 442 across the 26 of their papers we have counts for
18 papers · 1 filter
Understanding Hessian Alignment for Domain Generalization
Sobhan Hemati, Guojun Zhang, Amir Estiri +1
Out-of-distribution (OOD) generalization is a critical ability for deep learning models in many real-world scenarios including healthcare and autonomous vehicles. Recently, differe…
Understanding the Role of Layer Normalization in Label-Skewed Federated Learning
Guojun Zhang, Mahdi Beitollahi, Alex Bie +1
Layer normalization (LN) is a widely adopted deep learning technique especially in the era of foundation models. Recently, LN has been shown to be surprisingly effective in federat…
Mathematical Challenges in Deep Learning
Vahid Partovi Nia, Guojun Zhang, Ivan Kobyzev +8
Deep models are dominating the artificial intelligence (AI) industry since the ImageNet challenge in 2012. The size of deep models is increasing ever since, which brings new challe…
Private GANs, Revisited
Alex Bie, Gautam Kamath, Guojun Zhang
We show that the canonical approach for training differentially private GANs -- updating the discriminator with differentially private stochastic gradient descent (DPSGD) -- can yi…
Label Alignment Regularization for Distribution Shift
Ehsan Imani, Guojun Zhang, Runjia Li +4
Recent work has highlighted the label alignment property (LAP) in supervised learning, where the vector of all labels in the dataset is mostly in the span of the top few singular v…
DP-VAE: Differentially Private Pre-trained Variational Autoencoders
Dihong Jiang, Guojun Zhang, Mahdi Karami +3
Modern machine learning systems achieve great success when trained on large datasets. However, these datasets usually contain sensitive information (e.g. medical records, face imag…
Mitigating Data Heterogeneity in Federated Learning with Data Augmentation
Artur Back de Luca, Guojun Zhang, Xi Chen +1
Federated Learning (FL) is a prominent framework that enables training a centralized model while securing user privacy by fusing local, decentralized models. In this setting, one m…
Robust One Round Federated Learning with Predictive Space Bayesian Inference
Mohsin Hasan, Zehao Zhang, Kaiyang Guo +4
Making predictions robust is an important challenge. A separate challenge in federated learning (FL) is to reduce the number of communication rounds, particularly since doing so re…
Federated Bayesian Neural Regression: A Scalable Global Federated Gaussian Process
Haolin Yu, Kaiyang Guo, Mahdi Karami +3
In typical scenarios where the Federated Learning (FL) framework applies, it is common for clients to have insufficient training data to produce an accurate model. Thus, models tha…
Domain Adversarial Training: A Game Perspective
David Acuna, Marc T Law, Guojun Zhang +1
The dominant line of work in domain adaptation has focused on learning invariant representations using domain-adversarial training. In this paper, we interpret this approach from a…
Proportional Fairness in Federated Learning
Guojun Zhang, Saber Malekmohammadi, Xi Chen +1
With the increasingly broad deployment of federated learning (FL) systems in the real world, it is critical but challenging to ensure fairness in FL, i.e. reasonably satisfactory p…
f-Domain-Adversarial Learning: Theory and Algorithms
David Acuna, Guojun Zhang, Marc T. Law +1
Unsupervised domain adaptation is used in many machine learning applications where, during training, a model has access to unlabeled data in the target domain, and a related labele…
Quantifying and Improving Transferability in Domain Generalization
Guojun Zhang, Han Zhao, Yaoliang Yu +1
Out-of-distribution generalization is one of the key challenges when transferring a model from the lab to the real world. Existing efforts mostly focus on building invariant featur…
Newton-type Methods for Minimax Optimization
Guojun Zhang, Kaiwen Wu, Pascal Poupart +1
Differential games, in particular two-player sequential zero-sum games (a.k.a. minimax optimization), have been an important modeling tool in applied science and received renewed i…
Federated Learning Meets Multi-objective Optimization
Zeou Hu, Kiarash Shaloudegi, Guojun Zhang +1
Federated learning has emerged as a promising, massively distributed way to train a joint deep model over large amounts of edge devices while keeping private user data strictly on…
Optimality and Stability in Non-Convex Smooth Games
Guojun Zhang, Pascal Poupart, Yaoliang Yu
Convergence to a saddle point for convex-concave functions has been studied for decades, while recent years has seen a surge of interest in non-convex (zero-sum) smooth games, moti…
Convergence of Gradient Methods on Bilinear Zero-Sum Games
Guojun Zhang, Yaoliang Yu
Min-max formulations have attracted great attention in the ML community due to the rise of deep generative models and adversarial methods, while understanding the dynamics of gradi…
Comparing EM with GD in Mixture Models of Two Components
Guojun Zhang, Pascal Poupart, George Trimponias
The expectation-maximization (EM) algorithm has been widely used in minimizing the negative log likelihood (also known as cross entropy) of mixture models. However, little is under…