Online Learning with Gated Linear Networks

Abstract

This paper describes a family of probabilistic architectures designed for online learning under the logarithmic loss. Rather than relying on non-linear transfer functions, our method gains representational power by the use of data conditioning. We state under general conditions a learnable capacity theorem that shows this approach can in principle learn any bounded Borel-measurable function on a compact subset of euclidean space; the result is stronger than many universality results for connectionist architectures because we provide both the model and the learning procedure for which convergence is guaranteed.

40 pages

References in corpus (3)

• Exact solutions to the nonlinear dynamics of learning in deep linear neural networks
• Deep Online Convex Optimization with Gated Games
• Linear and Geometric Mixtures - Analysis

Cited by in corpus (9)

• Scalable agent alignment via reward modeling: a research direction
• Sample Efficient Adaptive Text-to-Speech
• DAWSON: A Domain Adaptive Few Shot Generation Framework
• Kernelized information bottleneck leads to biologically plausible 3-factor Hebbian learning in deep networks
• Associative Compression Networks for Representation Learning
• Online Learning in Contextual Bandits using Gated Linear Networks
• Gaussian Gated Linear Networks
• Product Kanerva Machines: Factorized Bayesian Memory
• A Combinatorial Perspective on Transfer Learning