Impact of Fully Connected Layers on Performance of Convolutional Neural Networks for Image Classification

Abstract

The Convolutional Neural Networks (CNNs), in domains like computer vision, mostly reduced the need for handcrafted features due to its ability to learn the problem-specific features from the raw input data. However, the selection of dataset-specific CNN architecture, which mostly performed by either experience or expertise is a time-consuming and error-prone process. To automate the process of learning a CNN architecture, this paper attempts at finding the relationship between Fully Connected (FC) layers with some of the characteristics of the datasets. The CNN architectures, and recently datasets also, are categorized as deep, shallow, wide, etc. This paper tries to formalize these terms along with answering the following questions. (i) What is the impact of deeper/shallow architectures on the performance of the CNN w.r.t. FC layers?, (ii) How the deeper/wider datasets influence the performance of CNN w.r.t. FC layers?, and (iii) Which kind of architecture (deeper/ shallower) is better suitable for which kind of (deeper/ wider) datasets. To address these findings, we have performed experiments with three CNN architectures having different depths. The experiments are conducted by varying the number of FC layers. We used four widely used datasets including CIFAR-10, CIFAR-100, Tiny ImageNet, and CRCHistoPhenotypes to justify our findings in the context of the image classification problem. The source code of this research is available at https://github.com/shabbeersh/Impact-of-FC-layers.

arXiv commentThis paper is accepted for publication in Neurocomputing Journal

References in corpus (9)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Neural Architecture Search with Reinforcement Learning
• Do Deep Nets Really Need to be Deep?
• On the Number of Linear Regions of Deep Neural Networks
• Impact of Fully Connected Layers on Performance of Convolutional Neural Networks for Image Classification
• The Expressive Power of Neural Networks: A View from the Width
• Learning Functions: When Is Deep Better Than Shallow
• RCCNet: An Efficient Convolutional Neural Network for Histological Routine Colon Cancer Nuclei Classification

Cited by in corpus (14)

• Impact of Fully Connected Layers on Performance of Convolutional Neural Networks for Image Classification
• AutoFCL: Automatically Tuning Fully Connected Layers for Handling Small Dataset
• Deep learning for multi-label classification of coral conditions in the Indo-Pacific via underwater photogrammetry
• Looking GLAMORous: Vehicle Re-Id in Heterogeneous Cameras Networks with Global and Local Attention
• Plant Leaf Disease Detection and Classification Using Deep Learning: A Review and A Proposed System on Bangladesh's Perspective
• DenseNet Models for Tiny ImageNet Classification
• Feature space approximation for kernel-based supervised learning
• Insights on Neural Representations for End-to-End Speech Recognition
• Deep Feature Response Discriminative Calibration
• Small, Accurate, and Fast Vehicle Re-ID on the Edge: the SAFR Approach
• Hyperspectral imaging with Raman scattered photons: A new paradigm in Raman analysis
• TrIM, Triangular Input Movement Systolic Array for Convolutional Neural Networks: Dataflow and Analytical Modelling
• Frequency Disentangled Residual Network
• Characterizing and Measuring the Similarity of Neural Networks with Persistent Homology