Overfitting has a limitation: a model-independent generalization gap bound based on Rényi entropy

Atsushi Suzuki, Jing Wang

Will further scaling up of machine learning models continue to bring success? A significant challenge in answering this question lies in understanding generalization gap, which is…