The Unreasonable Effectiveness of Fully-Connected Layers for Low-Data Regimes

Peter Kocsis, Peter Súkeník, Guillem Brasó +3

Convolutional neural networks were the standard for solving many computer vision tasks until recently, when Transformers of MLP-based architectures have started to show competitive…