A Survey of the Recent Architectures of Deep Convolutional Neural Networks

Abstract

Deep Convolutional Neural Network (CNN) is a special type of Neural Networks, which has shown exemplary performance on several competitions related to Computer Vision and Image Processing. Some of the exciting application areas of CNN include Image Classification and Segmentation, Object Detection, Video Processing, Natural Language Processing, and Speech Recognition. The powerful learning ability of deep CNN is primarily due to the use of multiple feature extraction stages that can automatically learn representations from the data. The availability of a large amount of data and improvement in the hardware technology has accelerated the research in CNNs, and recently interesting deep CNN architectures have been reported. Several inspiring ideas to bring advancements in CNNs have been explored, such as the use of different activation and loss functions, parameter optimization, regularization, and architectural innovations. However, the significant improvement in the representational capacity of the deep CNN is achieved through architectural innovations. Notably, the ideas of exploiting spatial and channel information, depth and width of architecture, and multi-path information processing have gained substantial attention. Similarly, the idea of using a block of layers as a structural unit is also gaining popularity. This survey thus focuses on the intrinsic taxonomy present in the recently reported deep CNN architectures and, consequently, classifies the recent innovations in CNN architectures into seven different categories. These seven categories are based on spatial exploitation, depth, multi-path, width, feature-map exploitation, channel boosting, and attention. Additionally, the elementary understanding of CNN components, current challenges, and applications of CNN are also provided.

Number of Pages: 70, Number of Figures: 11, Number of Tables: 11. Artif Intell Rev (2020)

References in corpus (11)

• Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks
• Deep Big Simple Neural Nets Excel on Handwritten Digit Recognition
• Show and Tell: Lessons learned from the 2015 MSCOCO Image Captioning Challenge
• Recent progress in semantic image segmentation
• A Convolutional Neural Network Neutrino Event Classifier
• GAN Lab: Understanding Complex Deep Generative Models using Interactive Visual Experimentation
• Sequential Deep Trajectory Descriptor for Action Recognition with Three-stream CNN
• A Taxonomy of Deep Convolutional Neural Nets for Computer Vision
• Script Identification in Natural Scene Image and Video Frame using Attention based Convolutional-LSTM Network
• A Review of Modularization Techniques in Artificial Neural Networks
• Effect of Depth and Width on Local Minima in Deep Learning

Cited by in corpus (63)

• Domain Adaptation for Medical Image Analysis: A Survey
• Mapping the NFT revolution: market trends, trade networks and visual features
• A survey of the Vision Transformers and their CNN-Transformer based Variants
• A Survey on Ensemble Learning under the Era of Deep Learning
• DeepCervix: A Deep Learning-based Framework for the Classification of Cervical Cells Using Hybrid Deep Feature Fusion Techniques
• P2T: Pyramid Pooling Transformer for Scene Understanding
• Hardware and Software Optimizations for Accelerating Deep Neural Networks: Survey of Current Trends, Challenges, and the Road Ahead
• Pervasive AI for IoT applications: A Survey on Resource-efficient Distributed Artificial Intelligence
• A review on Deep Reinforcement Learning for Fluid Mechanics
• A review on deep reinforcement learning for fluid mechanics: an update
• Current Trends in Deep Learning for Earth Observation: An Open-source Benchmark Arena for Image Classification
• General Inverse Design of Thin-Film Metamaterials With Convolutional Neural Networks
• A Transfer Learning Evaluation of Deep Neural Networks for Image Classification
• Exploring the Synergies of Hybrid CNNs and ViTs Architectures for Computer Vision: A survey
• LegoDNN: Block-grained Scaling of Deep Neural Networks for Mobile Vision
• Detecting Adversarial Examples by Input Transformations, Defense Perturbations, and Voting
• Fair comparison of skin detection approaches on publicly available datasets
• EPE-NAS: Efficient Performance Estimation Without Training for Neural Architecture Search
• Point-Cloud Deep Learning of Porous Media for Permeability Prediction
• Single-stream CNN with Learnable Architecture for Multi-source Remote Sensing Data
• Binary Neural Networks for Memory-Efficient and Effective Visual Place Recognition in Changing Environments
• Comprehensive Performance Evaluation of YOLOv12, YOLO11, YOLOv10, YOLOv9 and YOLOv8 on Detecting and Counting Fruitlet in Complex Orchard Environments
• Learn Fine-grained Adaptive Loss for Multiple Anatomical Landmark Detection in Medical Images
• Role of Data Augmentation Strategies in Knowledge Distillation for Wearable Sensor Data
• TreeSketchNet: From Sketch To 3D Tree Parameters Generation
• A Review of Emerging Research Directions in Abstract Visual Reasoning
• CODEBench: A Neural Architecture and Hardware Accelerator Co-Design Framework
• Automated Quality Control of Vacuum Insulated Glazing by Convolutional Neural Network Image Classification
• Fuzzy Convolution Neural Networks for Tabular Data Classification
• A Dimensionality Reduction Approach for Convolutional Neural Networks
• A Two-Stage Deep Learning Detection Classifier for the ATLAS Asteroid Survey
• Quaternion Convolutional Neural Networks: Current Advances and Future Directions
• OnDev-LCT: On-Device Lightweight Convolutional Transformers towards federated learning
• Deep Learning for Biomedical Image Reconstruction: A Survey
• Photonic kernel machine learning for ultrafast spectral analysis
• Non-linear Neurons with Human-like Apical Dendrite Activations
• Elephant Flows Detection Using Deep Neural Network, Convolutional Neural Network, Long Short Term Memory and Autoencoder
• CB-HVTNet: A channel-boosted hybrid vision transformer network for lymphocyte assessment in histopathological images
• Pulse shape discrimination using a convolutional neural network for organic liquid scintillator signals
• Quantum autoencoders for image classification
• SIRe-Networks: Convolutional Neural Networks Architectural Extension for Information Preservation via Skip/Residual Connections and Interlaced Auto-Encoders
• Generative Design of Physical Objects using Modular Framework
• Modeling & Evaluating the Performance of Convolutional Neural Networks for Classifying Steel Surface Defects
• Generative Adversarial Networks with Physical Evaluators for Spray Simulation of Pintle Injector
• A Study on the Impact of Data Augmentation for Training Convolutional Neural Networks in the Presence of Noisy Labels
• An Interaction-based Convolutional Neural Network (ICNN) Towards Better Understanding of COVID-19 X-ray Images
• A Novel Interaction-based Methodology Towards Explainable AI with Better Understanding of Pneumonia Chest X-ray Images
• Multiplier with Reduced Activities and Minimized Interconnect for Inner Product Arrays
• ALEN: A Dual-Approach for Uniform and Non-Uniform Low-Light Image Enhancement
• IoMT-based Automated Leukemia Classification using CNN and Higher Order Singular Value
• Identifying Galaxy Cluster Mergers with Deep Neural Networks using Idealized Compton-y and X-ray maps
• Triple-Stream Deep Feature Selection with Metaheuristic Optimization and Machine Learning for Multi-Stage Hypertensive Retinopathy Diagnosis
• A Machine Learning Approach to Integral Field Unit Spectroscopy Observations: III. Disentangling Multiple Components in Hii regions
• Ground Reaction Force Estimation via Time-aware Knowledge Distillation
• Bayesian and Convolutional Networks for Hierarchical Morphological Classification of Galaxies
• Learnable Adaptive Cosine Estimator (LACE) for Image Classification
• Temporal Parameter-free Deep Skinning of Animated Meshes
• Guiding the retraining of convolutional neural networks against adversarial inputs
• Hier-EgoPack: Hierarchical Egocentric Video Understanding with Diverse Task Perspectives
• ProBoost: a Boosting Method for Probabilistic Classifiers
• Selective Correlation Based Knowledge Distillation for Ground Reaction Force Estimation
• Reachability Is NP-Complete Even for the Simplest Neural Networks
• Learning Mechanically Driven Emergent Behavior with Message Passing Neural Networks