Learning Theory Approach to Minimum Error Entropy Criterion

Abstract

We consider the minimum error entropy (MEE) criterion and an empirical risk minimization learning algorithm in a regression setting. A learning theory approach is presented for this MEE algorithm and explicit error bounds are provided in terms of the approximation ability and capacity of the involved hypothesis space when the MEE scaling parameter is large. Novel asymptotic analysis is conducted for the generalization error associated with Renyi's entropy and a Parzen window function, to overcome technical difficulties arisen from the essential differences between the classical least squares problems and the MEE setting. A semi-norm and the involved symmetrized least squares error are introduced, which is related to some ranking algorithms.

References in corpus (1)

• 2004 IMS Medallion Lecture: Local Rademacher complexities and oracle inequalities in risk minimization

Cited by in corpus (9)

• A Statistical Learning Approach to Modal Regression
• Measuring Dependence with Matrix-based Entropy Functional
• Stability and Optimization Error of Stochastic Gradient Descent for Pairwise Learning
• On the Robustness of Regularized Pairwise Learning Methods Based on Kernels
• Universal Consistency and Robustness of Localized Support Vector Machines
• A Framework of Learning Through Empirical Gain Maximization
• New Insights into Learning with Correntropy Based Regression
• A short note on extension theorems and their connection to universal consistency in machine learning
• Consistency Analysis of an Empirical Minimum Error Entropy Algorithm