5 papers
Sketched Gaussian Mechanism for Private Federated Learning
Qiaobo Li, Zhijie Chen, Arindam Banerjee
Communication cost and privacy are two major considerations in federated learning (FL). For communication cost, gradient compression by sketching the clients' transmitted model upd…
Sketched Adaptive Federated Deep Learning: A Sharp Convergence Analysis
Zhijie Chen, Qiaobo Li, Arindam Banerjee
Combining gradient compression methods (e.g., CountSketch, quantization) and adaptive optimizers (e.g., Adam, AMSGrad) is a desirable goal in federated learning (FL), with potentia…
Loss Gradient Gaussian Width based Generalization and Optimization Guarantees
Arindam Banerjee, Qiaobo Li, Yingxue Zhou
Generalization and optimization guarantees on the population loss often rely on uniform convergence based analysis, typically based on the Rademacher complexity of the predictors.…
Optimization for Neural Operators can Benefit from Width
Pedro Cisneros-Velarde, Bhavesh Shrimali, Arindam Banerjee
Neural Operators that directly learn mappings between function spaces, such as Deep Operator Networks (DONs) and Fourier Neural Operators (FNOs), have received considerable attenti…
Optimization and Generalization Guarantees for Weight Normalization
Pedro Cisneros-Velarde, Zhijie Chen, Sanmi Koyejo +1
Weight normalization (WeightNorm) is widely used in practice for the training of deep neural networks and modern deep learning libraries have built-in implementations of it. In thi…