Generalization Error of -Divergence Stabilized Algorithms via Duality
arXiv:2502.14544
Abstract
The solution to empirical risk minimization with -divergence regularization (ERM-DR) is extended to constrained optimization problems, establishing conditions for equivalence between the solution and constraints. A dual formulation of ERM-DR is introduced, providing a computationally efficient method to derive the normalization function of the ERM-DR solution. This dual approach leverages the Legendre-Fenchel transform and the implicit function theorem, enabling explicit characterizations of the generalization error for general algorithms under mild conditions, and another for ERM-DR solutions.
This is new work for ISIT2025. arXiv admin note: text overlap with arXiv:2402.00501