12 citations · 31 across the 15 of their papers we have counts for
18 papers
Information-Theoretic Analysis of Next-Token Prediction under Markovian Data
Masoud Kavian, Abdellatif Zaidi, Milad Sefidgaran
We develop an information-theoretic framework for generalization in next-token prediction under temporally dependent data. We consider independent trajectories generated by finite-…
Tighter CMI-Based Generalization Bounds via Stochastic Projection and Quantization
Milad Sefidgaran, Kimia Nadjahi, Abdellatif Zaidi
In this paper, we leverage stochastic projection and lossy compression to establish new conditional mutual information (CMI) bounds on the generalization error of statistical learn…
Multiview Representation Learning via Distributed Joint Latent Space Structuring
Milad Sefidgaran, Piotr Krasnowski, Abdellatif Zaidi
We study distributed multiview representation learning, a problem in which clients each observe a distinct but possibly statistically correlated view. The clients independently…
Generalization Guarantees for Representation Learning via Data-Dependent Gaussian Mixture Priors
Milad Sefidgaran, Abdellatif Zaidi, Piotr Krasnowski
We establish in-expectation and tail bounds on the generalization error of representation learning type algorithms. The bounds are in terms of the relative entropy between the dist…
Heterogeneity Matters even More in Distributed Learning: Study from Generalization Perspective
Masoud Kavian, Romain Chor, Milad Sefidgaran +1
In this paper, we investigate the effect of data heterogeneity across clients on the performance of distributed learning systems, i.e., one-round Federated Learning, as measured by…
Minimal Communication-Cost Statistical Learning
Milad Sefidgaran, Abdellatif Zaidi, Piotr Krasnowski
A client device which has access to training data samples needs to obtain a statistical hypothesis or model and then to send it to a remote server. The client and the serve…
Minimum Description Length and Generalization Guarantees for Representation Learning
Milad Sefidgaran, Abdellatif Zaidi, Piotr Krasnowski
A major challenge in designing efficient statistical supervised learning algorithms is finding representations that perform well not only on available training samples but also on…
Lessons from Generalization Error Analysis of Federated Learning: You May Communicate Less Often!
Milad Sefidgaran, Romain Chor, Abdellatif Zaidi +1
We investigate the generalization error of statistical learning models in a Federated Learning (FL) setting. Specifically, we study the evolution of the generalization error with t…
More Communication Does Not Result in Smaller Generalization Error in Federated Learning
Romain Chor, Milad Sefidgaran, Abdellatif Zaidi
We study the generalization error of statistical learning models in a Federated Learning (FL) setting. Specifically, there are devices or clients, each holding an independent o…
Data-dependent Generalization Bounds via Variable-Size Compressibility
Milad Sefidgaran, Abdellatif Zaidi
In this paper, we establish novel data-dependent upper bounds on the generalization error through the lens of a "variable-size compressibility" framework that we introduce newly he…
Rate-Distortion Theoretic Bounds on Generalization Error for Distributed Learning
Milad Sefidgaran, Romain Chor, Abdellatif Zaidi
In this paper, we use tools from rate-distortion theory to establish new upper bounds on the generalization error of statistical distributed learning algorithms. Specifically, ther…
Rate-Distortion Theoretic Generalization Bounds for Stochastic Learning Algorithms
Milad Sefidgaran, Amin Gohari, Gaël Richard +1
Understanding generalization in modern machine learning settings has been one of the major challenges in statistical learning theory. In this context, recent years have witnessed t…
Heavy Tails in SGD and Compressibility of Overparametrized Neural Networks
Melih Barsbey, Milad Sefidgaran, Murat A. Erdogdu +2
Neural network compression techniques have become increasingly popular as they can drastically reduce the storage and computation requirements for very large networks. Recent empir…
Zero-Error Sum Modulo Two with a Common Observation
Milad Sefidgaran, Aslan Tchamkerten
This paper investigates the classical modulo two sum problem in source coding, but with a common observation: a transmitter observes , the other transmitter observes …
Lower Bound on the Capacity of the Continuous-Space SSFM Model of Optical Fiber
Milad Sefidgaran, Mansoor Yousefi
The capacity of a discrete-time model of optical fiber described by the split-step Fourier method (SSFM) as a function of the signal-to-noise ratio and the number of s…
Distributed Function Computation Over a Rooted Directed Tree
Milad Sefidgaran, Aslan Tchamkerten
This paper establishes the rate region for a class of source coding function computation setups where sources of information are available at the nodes of a tree and where a functi…
On Cooperation in Multi-Terminal Computation and Rate Distortion
Milad Sefidgaran, Aslan Tchamkerten
A receiver wants to compute a function of two correlated sources separately observed by two transmitters. One of the transmitters may send a possibly private message to the other t…
On Computing a Function of Correlated Sources
Milad Sefidgaran, Aslan Tchamkerten
A receiver wants to compute a function f of two correlated sources X and Y and side information Z. What is the minimum number of bits that needs to be communicated by each transmit…