Rethinking Layer-wise Feature Amounts in Convolutional Neural Network Architectures

Abstract

We characterize convolutional neural networks with respect to the relative amount of features per layer. Using a skew normal distribution as a parametrized framework, we investigate the common assumption of monotonously increasing feature-counts with higher layers of architecture designs. Our evaluation on models with VGG-type layers on the MNIST, Fashion-MNIST and CIFAR-10 image classification benchmarks provides evidence that motivates rethinking of our common assumption: architectures that favor larger early layers seem to yield better accuracy.

Accepted at the Critiquing and Correcting Trends in Machine Learning (CRACT) Workshop at the 32nd Conference on Neural Information Processing Systems (NeurIPS 2018)

References in corpus (2)

• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Neural Architecture Search with Reinforcement Learning