Automatic Speech Recognition using Advanced Deep Learning Approaches: A survey

Abstract

Recent advancements in deep learning (DL) have posed a significant challenge for automatic speech recognition (ASR). ASR relies on extensive training datasets, including confidential ones, and demands substantial computational and storage resources. Enabling adaptive systems improves ASR performance in dynamic environments. DL techniques assume training and testing data originate from the same domain, which is not always true. Advanced DL techniques like deep transfer learning (DTL), federated learning (FL), and reinforcement learning (RL) address these issues. DTL allows high-performance models using small yet related datasets, FL enables training on confidential data without dataset possession, and RL optimizes decision-making in dynamic environments, reducing computation costs. This survey offers a comprehensive review of DTL, FL, and RL-based ASR frameworks, aiming to provide insights into the latest developments and aid researchers and professionals in understanding the current challenges. Additionally, transformers, which are advanced DL techniques heavily used in proposed ASR frameworks, are considered in this survey for their ability to capture extensive dependencies in the input ASR sequence. The paper starts by presenting the background of DTL, FL, RL, and Transformers and then adopts a well-designed taxonomy to outline the state-of-the-art approaches. Subsequently, a critical analysis is conducted to identify the strengths and weaknesses of each framework. Additionally, a comparative study is presented to highlight the existing challenges, paving the way for future research opportunities.

References in corpus (7)

• Deep Transfer Learning for Automatic Speech Recognition: Towards Better Generalization
• Deep transfer learning for intrusion detection in industrial control networks: A comprehensive review
• Deep Learning for Steganalysis of Diverse Data Types: A review of methods, taxonomy, challenges and future directions
• Unsupervised Automatic Speech Recognition: A Review
• Lifelong Incremental Reinforcement Learning with Online Bayesian Inference
• A CTC Alignment-based Non-autoregressive Transformer for End-to-end Automatic Speech Recognition
• Group Communication with Context Codec for Lightweight Source Separation

Cited by in corpus (10)

• Transformers and Large Language Models for Efficient Intrusion Detection Systems: A Comprehensive Survey
• Adaptive Cyber-Attack Detection in IIoT Using Attention-Based LSTM-CNN Models
• Artificial Intelligence for Cochlear Implants: Review of Strategies, Challenges, and Perspectives
• Advanced Deep Learning and Large Language Models: Comprehensive Insights for Cancer Detection
• Machine Learning and Transformers for Thyroid Carcinoma Diagnosis: A Review
• Recent Advances in Transformer and Large Language Models for UAV Applications
• Transfer Learning-Based Deep Residual Learning for Speech Recognition in Clean and Noisy Environments
• Improving Pretrained YAMNet for Enhanced Speech Command Detection via Transfer Learning
• Enhancing Cochlear Implant Signal Coding with Scaled Dot-Product Attention
• Generative AI for Video Translation: A Scalable Architecture for Multilingual Video Conferencing