Regularized Deep Networks in Intelligent Transportation Systems: A Taxonomy and a Case Study

Abstract

Intelligent Transportation Systems (ITS) are much correlated with data science mechanisms. Among the different correlation branches, this paper focuses on the neural network learning models. Some of the considered models are shallow and they get some user-defined features and learn the relationship, while deep models extract the necessary features before learning by themselves. Both of these paradigms are utilized in the recent intelligent transportation systems (ITS) to support decision-making by the aid of different operations such as frequent patterns mining, regression, clustering, and classification. When these learners cannot generalize the results and just memorize the training samples, they fail to support the necessities. In these cases, the testing error is bigger than the training error. This phenomenon is addressed as overfitting in the literature. Because, this issue decreases the reliability of learning systems, in ITS applications, we cannot use such over-fitted machine learning models for different tasks such as traffic prediction, the signal controlling, safety applications, emergency responses, mode detection, driving evaluation, etc. Besides, deep learning models use a great number of hyper-parameters, the overfitting in deep models is more attention. To solve this problem, the regularized learning models can be followed. The aim of this paper is to review the approaches presented to regularize the overfitting in different categories of ITS studies. Then, we give a case study on driving safety that uses a regularized version of the convolutional neural network (CNN).

A review paper with 8 pages, 2 figures, and 2 tables, submitted to 18th International Conference on Traffic & Transportation Engineering, Tehran, February 25-27, 2020. Artificial Intelligence Review (2021)

References in corpus (36)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Deep Learning in Neural Networks: An Overview
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Conditional Generative Adversarial Nets
• Practical Bayesian Optimization of Machine Learning Algorithms
• Playing Atari with Deep Reinforcement Learning
• An overview of gradient descent optimization algorithms
• Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation
• Neural Architecture Search with Reinforcement Learning
• Spectral Normalization for Generative Adversarial Networks
• The Effectiveness of Data Augmentation in Image Classification using Deep Learning
• Empirical Evaluation of Rectified Activations in Convolutional Network
• Deep Convolutional Neural Networks and Data Augmentation for Environmental Sound Classification
• Learning to Reweight Examples for Robust Deep Learning
• Data Augmentation Generative Adversarial Networks
• Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architectures
• FractalNet: Ultra-Deep Neural Networks without Residuals
• DeepSaliency: Multi-Task Deep Neural Network Model for Salient Object Detection
• Manifold Mixup: Better Representations by Interpolating Hidden States
• Inferring transportation modes from GPS trajectories using a convolutional neural network
• Methods of Solving Ill-Posed Problems
• Batch Renormalization: Towards Reducing Minibatch Dependence in Batch-Normalized Models
• Overfitting Mechanism and Avoidance in Deep Neural Networks
• Physics-Driven Regularization of Deep Neural Networks for Enhanced Engineering Design and Analysis
• Normalization Propagation: A Parametric Technique for Removing Internal Covariate Shift in Deep Networks
• A Review of Modularization Techniques in Artificial Neural Networks
• Robust Convolutional Neural Networks under Adversarial Noise
• Data-Driven Color Augmentation Techniques for Deep Skin Image Analysis
• Towards Understanding Regularization in Batch Normalization
• GradAug: A New Regularization Method for Deep Neural Networks
• Forecasting Transportation Network Speed Using Deep Capsule Networks with Nested LSTM Models
• RDEC: Integrating Regularization into Deep Embedded Clustering for Imbalanced Datasets
• Fiedler Regularization: Learning Neural Networks with Graph Sparsity
• How Not to Give a FLOP: Combining Regularization and Pruning for Efficient Inference
• Adaptive Low-Rank Factorization to regularize shallow and deep neural networks
• Regularized Flexible Activation Function Combinations for Deep Neural Networks

Cited by in corpus (11)

• Physics-informed machine learning for building performance simulation-A review of a nascent field
• Deep Radiomic Analysis for Predicting Coronavirus Disease 2019 in Computerized Tomography and X-ray Images
• AgRegNet: A Deep Regression Network for Flower and Fruit Density Estimation, Localization, and Counting in Orchards
• MLExchange: A web-based platform enabling exchangeable machine learning workflows for scientific studies
• xEEGNet: Towards Explainable AI in EEG Dementia Classification
• Early Autism Diagnosis based on Path Signature and Siamese Unsupervised Feature Compressor
• Limitations of Data-Driven Spectral Reconstruction -- An Optics-Aware Analysis
• Adaptive Low-Rank Factorization to regularize shallow and deep neural networks
• Conformalised data synthesis
• GenEFT: Understanding Statics and Dynamics of Model Generalization via Effective Theory
• DUA-D2C: Dynamic Uncertainty Aware Method for Overfitting Remediation in Deep Learning