7 papers
Tighter CMI-Based Generalization Bounds via Stochastic Projection and Quantization
Milad Sefidgaran, Kimia Nadjahi, Abdellatif Zaidi
In this paper, we leverage stochastic projection and lossy compression to establish new conditional mutual information (CMI) bounds on the generalization error of statistical learn…
Heterogeneity Matters even More in Distributed Learning: Study from Generalization Perspective
Masoud Kavian, Romain Chor, Milad Sefidgaran +1
In this paper, we investigate the effect of data heterogeneity across clients on the performance of distributed learning systems, i.e., one-round Federated Learning, as measured by…
Multiview Representation Learning via Distributed Joint Latent Space Structuring
Milad Sefidgaran, Abdellatif Zaidi, Piotr Krasnowski
We study distributed multiview representation learning, a problem in which clients each observe a distinct but possibly statistically correlated view. The clients independently…
Generalization Guarantees for Representation Learning via Data-Dependent Gaussian Mixture Priors
Milad Sefidgaran, Abdellatif Zaidi, Piotr Krasnowski
We establish in-expectation and tail bounds on the generalization error of representation learning type algorithms. The bounds are in terms of the relative entropy between the dist…
Minimal Communication-Cost Statistical Learning
Milad Sefidgaran, Abdellatif Zaidi, Piotr Krasnowski
A client device which has access to training data samples needs to obtain a statistical hypothesis or model and then to send it to a remote server. The client and the serve…
Data-dependent Generalization Bounds via Variable-Size Compressibility
Milad Sefidgaran, Abdellatif Zaidi
In this paper, we establish novel data-dependent upper bounds on the generalization error through the lens of a "variable-size compressibility" framework that we introduce newly he…