Autoencoding with a Classifier System
arXiv:1910.10579 · doi:10.1109/TEVC.2021.3079320
Abstract
Autoencoders are data-specific compression algorithms learned automatically from examples. The predominant approach has been to construct single large global models that cover the domain. However, training and evaluating models of increasing size comes at the price of additional time and computational cost. Conditional computation, sparsity, and model pruning techniques can reduce these costs while maintaining performance. Learning classifier systems (LCS) are a framework for adaptively subdividing input spaces into an ensemble of simpler local approximations that together cover the domain. LCS perform conditional computation through the use of a population of individual gating/guarding components, each associated with a local approximation. This article explores the use of an LCS to adaptively decompose the input domain into a collection of small autoencoders where local solutions of different complexity may emerge. In addition to benefits in convergence time and computational cost, it is shown possible to reduce code size as well as the resulting decoder computational cost when compared with the global model equivalent.
References in corpus (10)
- Sequence to Sequence Learning with Neural Networks
- Neural Architecture Search: A Survey
- A Survey on Evolutionary Neural Architecture Search
- Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks
- A Particle Swarm Optimization-based Flexible Convolutional Auto-Encoder for Image Classification
- Channel Gating Neural Networks
- Evolving Unsupervised Deep Neural Networks for Learning Meaningful Representations
- Variational Mixture-of-Experts Autoencoders for Multi-Modal Deep Generative Models
- Evolutionary Stochastic Gradient Descent for Optimization of Deep Neural Networks
- Autoencoding with a Learning Classifier System: Initial Results