424 citations · 518 across the 28 of their papers we have counts for
4 papers · 1 filter
Information-Theoretic Analysis of Minimax Excess Risk
Hassan Hafez-Kolahi, Behrad Moniri, Shohreh Kasaei
Two main concepts studied in machine learning theory are generalization gap (difference between train and test error) and excess risk (difference between test error and the minimum…
Rate-Distortion Analysis of Minimum Excess Risk in Bayesian Learning
Hassan Hafez-Kolahi, Behrad Moniri, Shohreh Kasaei +1
In parametric Bayesian learning, a prior is assumed on the parameter which determines the distribution of samples. In this setting, Minimum Excess Risk (MER) is defined as the…
Do Compressed Representations Generalize Better?
Hassan Hafez-Kolahi, Shohreh Kasaei, Mahdiyeh Soleymani-Baghshah
One of the most studied problems in machine learning is finding reasonable constraints that guarantee the generalization of a learning algorithm. These constraints are usually expr…
Information Bottleneck and its Applications in Deep Learning
Hassan Hafez-Kolahi, Shohreh Kasaei
Information Theory (IT) has been used in Machine Learning (ML) from early days of this field. In the last decade, advances in Deep Neural Networks (DNNs) have led to surprising imp…