On-Line AdaTron Learning of Unlearnable Rules

Abstract

We study the on-line AdaTron learning of linearly non-separable rules by a simple perceptron. Training examples are provided by a perceptron with a non-monotonic transfer function which reduces to the usual monotonic relation in a certain limit. We find that, although the on-line AdaTron learning is a powerful algorithm for the learnable rule, it does not give the best possible generalization error for unlearnable problems. Optimization of the learning rate is shown to greatly improve the performance of the AdaTron algorithm, leading to the best possible generalization error for a wide range of the parameter which controls the shape of the transfer function.)

RevTeX 17 pages, 8 figures, to appear in Phys.Rev.E

Cited by in corpus (6)

• Analysis of ensemble learning using simple perceptrons based on online learning theory
• Analysis of on-line learning when a moving teacher goes around a true teacher
• Statistical Mechanics of On-line Learning when a Moving Teacher Goes around an Unlearnable True Teacher
• Statistical Mechanics of Time Domain Ensemble Learning
• Statistical Mechanics of Linear and Nonlinear Time-Domain Ensemble Learning
• Generalization ability of a perceptron with non-monotonic transfer function