A comprehensive review of Binary Neural Network

Abstract

Deep learning (DL) has recently changed the development of intelligent systems and is widely adopted in many real-life applications. Despite their various benefits and potentials, there is a high demand for DL processing in different computationally limited and energy-constrained devices. It is natural to study game-changing technologies such as Binary Neural Networks (BNN) to increase deep learning capabilities. Recently remarkable progress has been made in BNN since they can be implemented and embedded on tiny restricted devices and save a significant amount of storage, computation cost, and energy consumption. However, nearly all BNN acts trade with extra memory, computation cost, and higher performance. This article provides a complete overview of recent developments in BNN. This article focuses exclusively on 1-bit activations and weights 1-bit convolution networks, contrary to previous surveys in which low-bit works are mixed in. It conducted a complete investigation of BNN's development -from their predecessors to the latest BNN algorithms/techniques, presenting a broad design pipeline and discussing each module's variants. Along the way, it examines BNN (a) purpose: their early successes and challenges; (b) BNN optimization: selected representative works that contain essential optimization techniques; (c) deployment: open-source frameworks for BNN modeling and development; (d) terminal: efficient computing architectures and devices for BNN and (e) applications: diverse applications with BNN. Moreover, this paper discusses potential directions and future research opportunities in each section.

accepted by journal of Artificial Intelligence Review

References in corpus (7)

• Binary Neural Networks: A Survey
• BinaryBERT: Pushing the Limit of BERT Quantization
• Larq Compute Engine: Design, Benchmark, and Deploy State-of-the-Art Binarized Neural Networks
• BiFSMN: Binary Neural Network for Keyword Spotting
• Distribution-sensitive Information Retention for Accurate Binary Neural Network
• SiMaN: Sign-to-Magnitude Network Binarization
• BiFSMNv2: Pushing Binary Neural Networks for Keyword Spotting to Real-Network Performance

Cited by in corpus (5)

• A Comprehensive Review and a Taxonomy of Edge Machine Learning: Requirements, Paradigms, and Techniques
• MUCM-Net: A Mamba Powered UCM-Net for Skin Lesion Segmentation
• Taming Binarized Neural Networks and Mixed-Integer Programs
• Variational Inference for Quantum HyperNetworks
• Training Multi-Layer Binary Neural Networks With Local Binary Error Signals