Online Learning with Ensembles

Abstract

Supervised online learning with an ensemble of students randomized by the choice of initial conditions is analyzed. For the case of the perceptron learning rule, asymptotically the same improvement in the generalization error of the ensemble compared to the performance of a single student is found as in Gibbs learning. For more optimized learning rules, however, using an ensemble yields no improvement. This is explained by showing that for any learning rule $f$ a transform $\tilde{f}$ exists, such that a single student using $\tilde{f}$ has the same generalization behaviour as an ensemble of $f$-students.

8 pages, 1 figure. Submitted to J.Phys.A

References in corpus (1)

• Self-Averaging and On-line Learning

Cited by in corpus (10)

• Analysis of ensemble learning using simple perceptrons based on online learning theory
• Ensemble learning of linear perceptron; Online learning theory
• Statistical Mechanics of Online Learning for Ensemble Teachers
• Analysis of on-line learning when a moving teacher goes around a true teacher
• Neural Synchronization and Cryptography
• Statistical Mechanics of On-line Learning when a Moving Teacher Goes around an Unlearnable True Teacher
• Statistical Mechanics of Nonlinear On-line Learning for Ensemble Teachers
• On-line Learning of an Unlearnable True Teacher through Mobile Ensemble Teachers
• Statistical Mechanics of Time Domain Ensemble Learning
• Statistical Mechanics of Linear and Nonlinear Time-Domain Ensemble Learning