Ensemble learning of linear perceptron; Online learning theory

Abstract

Within the framework of on-line learning, we study the generalization error of an ensemble learning machine learning from a linear teacher perceptron. The generalization error achieved by an ensemble of linear perceptrons having homogeneous or inhomogeneous initial weight vectors is precisely calculated at the thermodynamic limit of a large number of input elements and shows rich behavior. Our main findings are as follows. For learning with homogeneous initial weight vectors, the generalization error using an infinite number of linear student perceptrons is equal to only half that of a single linear perceptron, and converges with that of the infinite case with O(1/K) for a finite number of K linear perceptrons. For learning with inhomogeneous initial weight vectors, it is advantageous to use an approach of weighted averaging over the output of the linear perceptrons, and we show the conditions under which the optimal weights are constant during the learning process. The optimal weights depend on only correlation of the initial weight vectors.

14 pages, 3 figures, submitted to Physical Review E

References in corpus (1)

• Analysis of ensemble learning using simple perceptrons based on online learning theory

Cited by in corpus (9)

• Analysis of dropout learning regarded as ensemble learning
• Statistical Mechanics of Online Learning for Ensemble Teachers
• Analysis of on-line learning when a moving teacher goes around a true teacher
• Statistical Mechanics of On-line Learning when a Moving Teacher Goes around an Unlearnable True Teacher
• Statistical Mechanics of Nonlinear On-line Learning for Ensemble Teachers
• Statistical Mechanics of Time Domain Ensemble Learning
• Optimization of the Asymptotic Property of Mutual Learning Involving an Integration Mechanism of Ensemble Learning
• Statistical Mechanics of Linear and Nonlinear Time-Domain Ensemble Learning
• Statistical Mechanics of Node-perturbation Learning with Noisy Baseline