Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1

Abstract

We introduce a method to train Binarized Neural Networks (BNNs) - neural networks with binary weights and activations at run-time. At training-time the binary weights and activations are used for computing the parameters gradients. During the forward pass, BNNs drastically reduce memory size and accesses, and replace most arithmetic operations with bit-wise operations, which is expected to substantially improve power-efficiency. To validate the effectiveness of BNNs we conduct two sets of experiments on the Torch7 and Theano frameworks. On both, BNNs achieved nearly state-of-the-art results over the MNIST, CIFAR-10 and SVHN datasets. Last but not least, we wrote a binary matrix multiplication GPU kernel with which it is possible to run our MNIST BNN 7 times faster than with an unoptimized GPU kernel, without suffering any loss in classification accuracy. The code for training and running our BNNs is available on-line.

arXiv comment11 pages and 3 figures

References in corpus (21)

• Adam: A Method for Stochastic Optimization
• Sequence to Sequence Learning with Neural Networks
• FitNets: Hints for Thin Deep Nets
• BinaryConnect: Training Deep Neural Networks with binary weights during propagations
• Expectation Propagation for approximate Bayesian inference
• Going Deeper with Convolutions
• Theano: new features and speed improvements
• Deeply-Supervised Nets
• Compressing Deep Convolutional Networks using Vector Quantization
• Deep Learning using Linear Support Vector Machines
• Training deep neural networks with low precision multiplications
• Spatially-sparse convolutional neural networks
• Pylearn2: a machine learning research library
• Bitwise Neural Networks
• Neural Networks with Few Multiplications
• Resiliency of Deep Neural Networks under Quantization
• Subdominant Dense Clusters Allow for Simple Learning and High Computational Performance in Neural Networks with Discrete Synapses
• Generalizing Pooling Functions in Convolutional Neural Networks: Mixed, Gated, and Tree
• Reduced-Precision Strategies for Bounded Memory in Deep Neural Nets
• Estimating or Propagating Gradients Through Stochastic Neurons
• Training Binary Multilayer Neural Networks for Image Classification using Expectation Backpropagation

Cited by in corpus (556)

• DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
• AMC: AutoML for Model Compression and Acceleration on Mobile Devices
• Learning to Communicate with Deep Multi-Agent Reinforcement Learning
• FINN: A Framework for Fast, Scalable Binarized Neural Network Inference
• Deep Face Recognition: A Survey
• A Survey of Model Compression and Acceleration for Deep Neural Networks
• Ternary Weight Networks
• Pruning Filters for Efficient ConvNets
• FPGA-based Accelerators of Deep Learning Networks for Learning and Classification: A Review
• FedALA: Adaptive Local Aggregation for Personalized Federated Learning
• Artificial neural networks for neuroscientists: A primer
• Convolutional Neural Networks using Logarithmic Data Representation
• Channel Pruning for Accelerating Very Deep Neural Networks
• Like What You Like: Knowledge Distill via Neuron Selectivity Transfer
• Bayesian Compression for Deep Learning
• Distilling Task-Specific Knowledge from BERT into Simple Neural Networks
• Recent Advances in Convolutional Neural Networks
• Synaptic Plasticity Dynamics for Deep Continuous Local Learning (DECOLLE)
• Hardware-oriented Approximation of Convolutional Neural Networks
• Learning Efficient Convolutional Networks through Network Slimming
• MCUNet: Tiny Deep Learning on IoT Devices
• Hierarchical Multiscale Recurrent Neural Networks
• Demystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency Analysis
• Prospects and applications of photonic neural networks
• Flexpoint: An Adaptive Numerical Format for Efficient Training of Deep Neural Networks
• XONN: XNOR-based Oblivious Deep Neural Network Inference
• Learning Multimodal Graph-to-Graph Translation for Molecular Optimization
• Distilled Siamese Networks for Visual Tracking
• Surrogate Gradient Learning in Spiking Neural Networks
• Unreasonable Effectiveness of Learning Neural Networks: From Accessible States and Robust Ensembles to Basic Algorithmic Schemes
• An Entropy-based Pruning Method for CNN Compression
• Deep Identity-aware Transfer of Facial Attributes
• Dynamic Network Surgery for Efficient DNNs
• Enable Deep Learning on Mobile Devices: Methods, Systems, and Applications
• XNOR Neural Engine: a Hardware Accelerator IP for 21.6 fJ/op Binary Neural Network Inference
• IMAC: In-memory multi-bit Multiplication andACcumulation in 6T SRAM Array
• ALWANN: Automatic Layer-Wise Approximation of Deep Neural Network Accelerators without Retraining
• Optical Reservoir Computing using multiple light scattering for chaotic systems prediction
• Edge Intelligence: Architectures, Challenges, and Applications
• Compressing Neural Networks using the Variational Information Bottleneck
• XNOR-Net++: Improved Binary Neural Networks
• Exploring the Connection Between Binary and Spiking Neural Networks
• DeepCABAC: A Universal Compression Algorithm for Deep Neural Networks
• Sequence-Level Knowledge Distillation
• Deconstructing Lottery Tickets: Zeros, Signs, and the Supermask
• Rethinking floating point for deep learning
• Theory and Tools for the Conversion of Analog to Spiking Convolutional Neural Networks
• Ristretto: Hardware-Oriented Approximation of Convolutional Neural Networks
• Training Quantized Nets: A Deeper Understanding
• The Power of Sparsity in Convolutional Neural Networks
• Paraphrasing Complex Network: Network Compression via Factor Transfer
• ACNet: Strengthening the Kernel Skeletons for Powerful CNN via Asymmetric Convolution Blocks
• Deep Convolutional Neural Network Inference with Floating-point Weights and Fixed-point Activations
• BS4NN: Binarized Spiking Neural Networks with Temporal Coding and Learning
• SkyNet: a Hardware-Efficient Method for Object Detection and Tracking on Embedded Systems
• NeuralPower: Predict and Deploy Energy-Efficient Convolutional Neural Networks
• Embedded Binarized Neural Networks
• A Low Effort Approach to Structured CNN Design Using PCA
• Deep neural networks are robust to weight binarization and other non-linear distortions
• Learning N:M Fine-grained Structured Sparse Neural Networks From Scratch
• NSGA-Net: Neural Architecture Search using Multi-Objective Genetic Algorithm
• Training Binary Neural Networks with Real-to-Binary Convolutions
• Autonomous Probabilistic Coprocessing with Petaflips per Second
• Rotated Binary Neural Network
• A novel channel pruning method for deep neural network compression
• AddNet: Deep Neural Networks Using FPGA-Optimized Multipliers
• Latent Weights Do Not Exist: Rethinking Binarized Neural Network Optimization
• Deep Learning with Low Precision by Half-wave Gaussian Quantization
• Fast differentiable DNA and protein sequence optimization for molecular design
• Ternary Neural Networks with Fine-Grained Quantization
• Understanding Straight-Through Estimator in Training Activation Quantized Neural Nets
• A neural network memory prefetcher using semantic locality
• Security and Privacy Issues in Deep Learning
• TAPAS: Tricks to Accelerate (encrypted) Prediction As a Service
• Optimizing Speech Recognition For The Edge
• Structured Probabilistic Pruning for Convolutional Neural Network Acceleration
• Bi-GCN: Binary Graph Convolutional Network
• Differentiable Soft Quantization: Bridging Full-Precision and Low-Bit Neural Networks
• Overcoming Long-term Catastrophic Forgetting through Adversarial Neural Pruning and Synaptic Consolidation
• Trained Quantization Thresholds for Accurate and Efficient Fixed-Point Inference of Deep Neural Networks
• Compacting Deep Neural Networks for Internet of Things: Methods and Applications
• Long short-term memory and learning-to-learn in networks of spiking neurons
• DFTerNet: Towards 2-bit Dynamic Fusion Networks for Accurate Human Activity Recognition
• HAQ: Hardware-Aware Automated Quantization with Mixed Precision
• MEAL V2: Boosting Vanilla ResNet-50 to 80%+ Top-1 Accuracy on ImageNet without Tricks
• Lightweight Neural Architecture Search for Temporal Convolutional Networks at the Edge
• On-Device Machine Learning: An Algorithms and Learning Theory Perspective
• Constructing Energy-efficient Mixed-precision Neural Networks through Principal Component Analysis for Edge Intelligence
• Efficient Processing of Deep Neural Networks: A Tutorial and Survey
• TernaryNet: Faster Deep Model Inference without GPUs for Medical 3D Segmentation using Sparse and Binary Convolutions
• Texture Memory-Augmented Deep Patch-Based Image Inpainting
• ProjectionNet: Learning Efficient On-Device Deep Networks Using Neural Projections
• Delta Networks for Optimized Recurrent Network Computation
• ExpandNets: Linear Over-parameterization to Train Compact Convolutional Networks
• Towards Binary-Valued Gates for Robust LSTM Training
• BlockDrop: Dynamic Inference Paths in Residual Networks
• Improved training of binary networks for human pose estimation and image recognition
• DeepIoT: Compressing Deep Neural Network Structures for Sensing Systems with a Compressor-Critic Framework
• Hierarchical binary CNNs for landmark localization with limited resources
• EasyQuant: Post-training Quantization via Scale Optimization
• A Stochastic-Computing based Deep Learning Framework using Adiabatic Quantum-Flux-Parametron SuperconductingTechnology
• Binary Single-dimensional Convolutional Neural Network for Seizure Prediction
• ShiftAddNet: A Hardware-Inspired Deep Network
• Skip RNN: Learning to Skip State Updates in Recurrent Neural Networks
• More is Less: A More Complicated Network with Less Inference Complexity
• Complexity-Driven CNN Compression for Resource-constrained Edge AI
• SEALion: a Framework for Neural Network Inference on Encrypted Data
• Low-Memory Neural Network Training: A Technical Report
• A Learning Framework for n-bit Quantized Neural Networks toward FPGAs
• Defensive Quantization: When Efficiency Meets Robustness
• Towards Efficient Training for Neural Network Quantization
• Communication and Computation Reduction for Split Learning using Asynchronous Training
• Depthwise Convolution is All You Need for Learning Multiple Visual Domains
• Gradient Centralization: A New Optimization Technique for Deep Neural Networks
• BinaryDuo: Reducing Gradient Mismatch in Binary Activation Network by Coupling Binary Activations
• DSConv: Efficient Convolution Operator
• Controlling Information Capacity of Binary Neural Network
• Branch Prediction Is Not a Solved Problem: Measurements, Opportunities, and Future Directions
• FINN-R: An End-to-End Deep-Learning Framework for Fast Exploration of Quantized Neural Networks
• Musical Chair: Efficient Real-Time Recognition Using Collaborative IoT Devices
• Two-Bit Networks for Deep Learning on Resource-Constrained Embedded Devices
• Data-Driven Sparse Structure Selection for Deep Neural Networks
• ReActNet: Towards Precise Binary Neural Network with Generalized Activation Functions
• Understanding Neural Networks and Individual Neuron Importance via Information-Ordered Cumulative Ablation
• Binary Neural Networks for Memory-Efficient and Effective Visual Place Recognition in Changing Environments
• Distributed Learning in Wireless Networks: Recent Progress and Future Challenges
• IGCV3: Interleaved Low-Rank Group Convolutions for Efficient Deep Neural Networks
• Adaptive Neural Network-Based Approximation to Accelerate Eulerian Fluid Simulation
• Memory-Driven Mixed Low Precision Quantization For Enabling Deep Network Inference On Microcontrollers
• Learning Accurate Low-Bit Deep Neural Networks with Stochastic Quantization
• SomnNET: An SpO2 Based Deep Learning Network for Sleep Apnea Detection in Smartwatches
• Communication-Efficient Edge AI: Algorithms and Systems
• DeepHunter: Hunting Deep Neural Network Defects via Coverage-Guided Fuzzing
• Word2Bits - Quantized Word Vectors
• Self-Binarizing Networks
• Reservoirs learn to learn
• GCNv2: Efficient Correspondence Prediction for Real-Time SLAM
• Knowledge Distillation from Internal Representations
• Mixed Low-precision Deep Learning Inference using Dynamic Fixed Point
• Exploring the Function Space of Deep-Learning Machines
• Scalable Rule-Based Representation Learning for Interpretable Classification
• On the role of synaptic stochasticity in training low-precision neural networks
• AdaptivFloat: A Floating-point based Data Type for Resilient Deep Learning Inference
• Packing Sparse Convolutional Neural Networks for Efficient Systolic Array Implementations: Column Combining Under Joint Optimization
• Attacking Binarized Neural Networks
• Lightweight Residual Densely Connected Convolutional Neural Network
• Optimally Scheduling CNN Convolutions for Efficient Memory Access
• Model Slicing for Supporting Complex Analytics with Elastic Inference Cost and Resource Constraints
• QGAN: Quantized Generative Adversarial Networks
• Deep Learning for Real-Time Crime Forecasting and its Ternarization
• Intelligence Beyond the Edge: Inference on Intermittent Embedded Systems
• YOLoC: DeploY Large-Scale Neural Network by ROM-based Computing-in-Memory using ResiduaL Branch on a Chip
• Hardware-Robust In-RRAM-Computing for Object Detection
• Differentiable Model Compression via Pseudo Quantization Noise
• Collaborative Execution of Deep Neural Networks on Internet of Things Devices
• On the Adversarial Robustness of Quantized Neural Networks
• Fixed-point Quantization of Convolutional Neural Networks for Quantized Inference on Embedded Platforms
• Learning Convolutional Networks for Content-weighted Image Compression
• Performance Guaranteed Network Acceleration via High-Order Residual Quantization
• Projection Convolutional Neural Networks for 1-bit CNNs via Discrete Back Propagation
• Arrhythmia Classifier Based on Ultra-Lightweight Binary Neural Network
• Binarized Neural Networks on the ImageNet Classification Task
• Weight-dependent Gates for Network Pruning
• Efficient Visual Recognition with Deep Neural Networks: A Survey on Recent Advances and New Directions
• To compress or not to compress: Understanding the Interactions between Adversarial Attacks and Neural Network Compression
• Robust Sparse Regularization: Simultaneously Optimizing Neural Network Robustness and Compactness
• HashGAN:Attention-aware Deep Adversarial Hashing for Cross Modal Retrieval
• AdaBits: Neural Network Quantization with Adaptive Bit-Widths
• Post-Training 4-bit Quantization on Embedding Tables
• I-BERT: Integer-only BERT Quantization
• Knowledge Distillation for Feature Extraction in Underwater VSLAM
• Reverse Derivative Ascent: A Categorical Approach to Learning Boolean Circuits
• Training Binary Neural Networks using the Bayesian Learning Rule
• FQ-Conv: Fully Quantized Convolution for Efficient and Accurate Inference
• Precision Gating: Improving Neural Network Efficiency with Dynamic Dual-Precision Activations
• Hardware-Centric AutoML for Mixed-Precision Quantization
• A Tandem Learning Rule for Effective Training and Rapid Inference of Deep Spiking Neural Networks
• KTAN: Knowledge Transfer Adversarial Network
• Binary Ensemble Neural Network: More Bits per Network or More Networks per Bit?
• On Quantizing Implicit Neural Representations
• Quantization and Training of Low Bit-Width Convolutional Neural Networks for Object Detection
• An Encoding Framework for Binarized Images using HyperDimensional Computing
• Adaptive Precision CNN Accelerator Using Radix-X Parallel Connected Memristor Crossbars
• Neural Network-Hardware Co-design for Scalable RRAM-based BNN Accelerators
• The Architectural Implications of Facebook's DNN-based Personalized Recommendation
• A Configurable BNN ASIC using a Network of Programmable Threshold Logic Standard Cells
• FD-MobileNet: Improved MobileNet with a Fast Downsampling Strategy
• Centripetal SGD for Pruning Very Deep Convolutional Networks with Complicated Structure
• Progressive Tandem Learning for Pattern Recognition with Deep Spiking Neural Networks
• One-Shot Pruning of Recurrent Neural Networks by Jacobian Spectrum Evaluation
• SEP-Nets: Small and Effective Pattern Networks
• On Periodic Functions as Regularizers for Quantization of Neural Networks
• Search for Better Students to Learn Distilled Knowledge
• Orchestrating the Development Lifecycle of Machine Learning-Based IoT Applications: A Taxonomy and Survey
• Neural networks on microcontrollers: saving memory at inference via operator reordering
• Trained Rank Pruning for Efficient Deep Neural Networks
• Knowledge Squeezed Adversarial Network Compression
• A Study of Deep Learning Robustness Against Computation Failures
• Comprehensive SNN Compression Using ADMM Optimization and Activity Regularization
• A Survey on Impact of Transient Faults on BNN Inference Accelerators
• Benchmarking Quantized Neural Networks on FPGAs with FINN
• Improving Branch Prediction By Modeling Global History with Convolutional Neural Networks
• A Binary Convolutional Encoder-decoder Network for Real-time Natural Scene Text Processing
• BoolNet: Minimizing The Energy Consumption of Binary Neural Networks
• Blended Coarse Gradient Descent for Full Quantization of Deep Neural Networks
• Multi-Prize Lottery Ticket Hypothesis: Finding Accurate Binary Neural Networks by Pruning A Randomly Weighted Network
• The Convergence of Machine Learning and Communications
• Quantized Densely Connected U-Nets for Efficient Landmark Localization
• Sauron U-Net: Simple automated redundancy elimination in medical image segmentation via filter pruning
• An Embarrassingly Simple Approach for Knowledge Distillation
• Towards Evolutional Compression
• Pruning In Time (PIT): A Lightweight Network Architecture Optimizer for Temporal Convolutional Networks
• Towards Low-Latency Energy-Efficient Deep SNNs via Attention-Guided Compression
• LCNN: Lookup-based Convolutional Neural Network
• FxP-QNet: A Post-Training Quantizer for the Design of Mixed Low-Precision DNNs with Dynamic Fixed-Point Representation
• Equivalent and Approximate Transformations of Deep Neural Networks
• Nanophotonic Media for Artificial Neural Inference
• BiFSMN: Binary Neural Network for Keyword Spotting
• Towards Compact ConvNets via Structure-Sparsity Regularized Filter Pruning
• Efficient Stochastic Inference of Bitwise Deep Neural Networks
• AC/DC: Alternating Compressed/DeCompressed Training of Deep Neural Networks
• CPT: Efficient Deep Neural Network Training via Cyclic Precision
• Differentiable Fine-grained Quantization for Deep Neural Network Compression
• PAMS: Quantized Super-Resolution via Parameterized Max Scale
• MOBIUS: Model-Oblivious Binarized Neural Networks
• Gradient Scheduling with Global Momentum for Non-IID Data Distributed Asynchronous Training
• Training High-Performance and Large-Scale Deep Neural Networks with Full 8-bit Integers
• Full deep neural network training on a pruned weight budget
• Sparse Weight Activation Training
• Rethinking Class-Discrimination Based CNN Channel Pruning
• Joint Pruning & Quantization for Extremely Sparse Neural Networks
• Patch Slimming for Efficient Vision Transformers
• TernaryBERT: Distillation-aware Ultra-low Bit BERT
• Learning on Hardware: A Tutorial on Neural Network Accelerators and Co-Processors
• Espresso: Efficient Forward Propagation for BCNNs
• Rethinking Machine Learning Development and Deployment for Edge Devices
• TKD: Temporal Knowledge Distillation for Active Perception
• BinaryRelax: A Relaxation Approach For Training Deep Neural Networks With Quantized Weights
• RePose: Learning Deep Kinematic Priors for Fast Human Pose Estimation
• SenSeNet: Neural Keyphrase Generation with Document Structure
• Efficient Inference of CNNs via Channel Pruning
• Any-Precision Deep Neural Networks
• Structured Pruning for Efficient ConvNets via Incremental Regularization
• Tango: A Deep Neural Network Benchmark Suite for Various Accelerators
• Reproducible Floating-Point Aggregation in RDBMSs
• LUTNet: Rethinking Inference in FPGA Soft Logic
• An Integer Programming Approach to Deep Neural Networks with Binary Activation Functions
• Memristor-based Deep Convolution Neural Network: A Case Study
• Simultaneously Optimizing Weight and Quantizer of Ternary Neural Network using Truncated Gaussian Approximation
• Boosting Binary Masks for Multi-Domain Learning through Affine Transformations
• Regularizing by the Variance of the Activations' Sample-Variances
• Learning Versatile Convolution Filters for Efficient Visual Recognition
• A Heterogeneous Parallel Non-von Neumann Architecture System for Accurate and Efficient Machine Learning Molecular Dynamics
• OMPQ: Orthogonal Mixed Precision Quantization
• Some open questions on morphological operators and representations in the deep learning era
• Organ Segmentation From Full-size CT Images Using Memory-Efficient FCN
• Depth-wise Decomposition for Accelerating Separable Convolutions in Efficient Convolutional Neural Networks
• Propagating Asymptotic-Estimated Gradients for Low Bitwidth Quantized Neural Networks
• Learning Filter Basis for Convolutional Neural Network Compression
• Automating Generation of Low Precision Deep Learning Operators
• Deep Spiking Neural Networks for Large Vocabulary Automatic Speech Recognition
• Collaborative Distillation for Ultra-Resolution Universal Style Transfer
• BrainSlug: Transparent Acceleration of Deep Learning Through Depth-First Parallelism
• Hu-Fu: Hardware and Software Collaborative Attack Framework against Neural Networks
• Accelerating Binarized Neural Networks via Bit-Tensor-Cores in Turing GPUs
• Towards Lossless Binary Convolutional Neural Networks Using Piecewise Approximation
• Reservoir Computing Hardware with Cellular Automata
• Mix and Match: A Novel FPGA-Centric Deep Neural Network Quantization Framework
• Improving Deep Representation Learning via Auxiliary Learnable Target Coding
• Ternary Residual Networks
• Differentiable Dynamic Quantization with Mixed Precision and Adaptive Resolution
• Towards High Performance Video Object Detection
• Scaling Binarized Neural Networks on Reconfigurable Logic
• BiQGEMM: Matrix Multiplication with Lookup Table For Binary-Coding-based Quantized DNNs
• Ordering Chaos: Memory-Aware Scheduling of Irregularly Wired Neural Networks for Edge Devices
• Conditionally Deep Hybrid Neural Networks Across Edge and Cloud
• Taming Binarized Neural Networks and Mixed-Integer Programs
• BoMaNet: Boolean Masking of an Entire Neural Network
• WrapNet: Neural Net Inference with Ultra-Low-Resolution Arithmetic
• Nonuniform-to-Uniform Quantization: Towards Accurate Quantization via Generalized Straight-Through Estimation
• GXNOR-Net: Training deep neural networks with ternary weights and activations without full-precision memory under a unified discretization framework
• Real-time Memory Efficient Large-pose Face Alignment via Deep Evolutionary Network
• Efficient Contextualized Representation: Language Model Pruning for Sequence Labeling
• Neural Weighted A*: Learning Graph Costs and Heuristics with Differentiable Anytime A*
• HarDNet: A Low Memory Traffic Network
• Privacy-Preserving Image Classification in the Local Setting
• Tensor Decomposition for Compressing Recurrent Neural Network
• Learning Student Networks via Feature Embedding
• Evolutionary Bin Packing for Memory-Efficient Dataflow Inference Acceleration on FPGA
• Recursive Binary Neural Network Learning Model with 2.28b/Weight Storage Requirement
• Data-Free Learning of Student Networks
• Design Flow of Accelerating Hybrid Extremely Low Bit-width Neural Network in Embedded FPGA
• Low-Power Computer Vision: Status, Challenges, Opportunities
• Deep Learning in the Automotive Industry: Recent Advances and Application Examples
• Artificial neural networks condensation: A strategy to facilitate adaption of machine learning in medical settings by reducing computational burden
• Learning Content-Weighted Deep Image Compression
• Exploiting Channel Similarity for Accelerating Deep Convolutional Neural Networks
• End-to-End Learned Image Compression with Quantized Weights and Activations
• All-optical neuromorphic binary convolution with a spiking VCSEL neuron for image gradient magnitudes
• Adaptive Precision Training (AdaPT): A dynamic fixed point quantized training approach for DNNs
• BARS: Joint Search of Cell Topology and Layout for Accurate and Efficient Binary ARchitectures
• QuTiBench: Benchmarking Neural Networks on Heterogeneous Hardware
• Phoenix: A Low-Precision Floating-Point Quantization Oriented Architecture for Convolutional Neural Networks
• Verification of Binarized Neural Networks via Inter-Neuron Factoring
• A Bop and Beyond: A Second Order Optimizer for Binarized Neural Networks
• B-DCGAN:Evaluation of Binarized DCGAN for FPGA
• HYPER-SNN: Towards Energy-efficient Quantized Deep Spiking Neural Networks for Hyperspectral Image Classification
• RBCN: Rectified Binary Convolutional Networks for Enhancing the Performance of 1-bit DCNNs
• Mixed-Precision Quantized Neural Network with Progressively Decreasing Bitwidth For Image Classification and Object Detection
• Dependency Aware Filter Pruning
• Training Progressively Binarizing Deep Networks Using FPGAs
• Learning Context-Based Non-local Entropy Modeling for Image Compression
• Toward Efficient Convolutional Neural Networks With Structured Ternary Patterns
• Theano-MPI: a Theano-based Distributed Training Framework
• A&B BNN: Add&Bit-Operation-Only Hardware-Friendly Binary Neural Network
• CNN Acceleration by Low-rank Approximation with Quantized Factors
• Automatic Neural Network Compression by Sparsity-Quantization Joint Learning: A Constrained Optimization-based Approach
• A GPU-Outperforming FPGA Accelerator Architecture for Binary Convolutional Neural Networks
• Learning Architectures for Binary Networks
• No Multiplication? No Floating Point? No Problem! Training Networks for Efficient Inference
• The Future of Computing: Bits + Neurons + Qubits
• A Survey of Mobile Computing for the Visually Impaired
• FFT-Based Deep Learning Deployment in Embedded Systems
• DeepLight: Deep Lightweight Feature Interactions for Accelerating CTR Predictions in Ad Serving
• Quantization of Deep Neural Networks for Accurate Edge Computing
• A Deep Unsupervised Feature Learning Spiking Neural Network with Binarized Classification Layers for EMNIST Classification using SpykeFlow
• Neural Pruning via Growing Regularization
• Synthesis and Pruning as a Dynamic Compression Strategy for Efficient Deep Neural Networks
• Fully Learnable Group Convolution for Acceleration of Deep Neural Networks
• Manipulating Identical Filter Redundancy for Efficient Pruning on Deep and Complicated CNN
• MSP: An FPGA-Specific Mixed-Scheme, Multi-Precision Deep Neural Network Quantization Framework
• Deep counter networks for asynchronous event-based processing
• Binarized Neural Architecture Search for Efficient Object Recognition
• Binary Complex Neural Network Acceleration on FPGA
• Inference with Hybrid Bio-hardware Neural Networks
• PIMBALL: Binary Neural Networks in Spintronic Memory
• Predicting Adversarial Examples with High Confidence
• Matrix and tensor decompositions for training binary neural networks
• Alternating Direction Method of Multipliers for Quantization
• Dynamic Sparse Graph for Efficient Deep Learning
• Dataflow-based Joint Quantization of Weights and Activations for Deep Neural Networks
• Do We Need Fully Connected Output Layers in Convolutional Networks?
• FDDWNet: A Lightweight Convolutional Neural Network for Real-time Sementic Segmentation
• LCP: A Low-Communication Parallelization Method for Fast Neural Network Inference in Image Recognition
• Distilling portable Generative Adversarial Networks for Image Translation
• TinyCNN: A Tiny Modular CNN Accelerator for Embedded FPGA
• Sparse Quantized Spectral Clustering
• Big-Little Adaptive Neural Networks on Low-Power Near-Subthreshold Processors
• Automated flow for compressing convolution neural networks for efficient edge-computation with FPGA
• SparseNN: An Energy-Efficient Neural Network Accelerator Exploiting Input and Output Sparsity
• Toward Computation and Memory Efficient Neural Network Acoustic Models with Binary Weights and Activations
• Table-Based Neural Units: Fully Quantizing Networks for Multiply-Free Inference
• Progressive Learning of Low-Precision Networks
• Streaming Voice Query Recognition using Causal Convolutional Recurrent Neural Networks
• Towards Explainable Bit Error Tolerance of Resistive RAM-Based Binarized Neural Networks
• Distributed Low Precision Training Without Mixed Precision
• Sampling-Free Learning of Bayesian Quantized Neural Networks
• Binarized Neural Architecture Search
• Training convolutional neural networks with cheap convolutions and online distillation
• Softmax Tempering for Training Neural Machine Translation Models
• An FPGA Accelerated Method for Training Feed-forward Neural Networks Using Alternating Direction Method of Multipliers and LSMR
• NASB: Neural Architecture Search for Binary Convolutional Neural Networks
• Joint Multi-User DNN Partitioning and Computational Resource Allocation for Collaborative Edge Intelligence
• Multi-Tones' Phase Coding (MTPC) of Interaural Time Difference by Spiking Neural Network
• AnalogNet: Convolutional Neural Network Inference on Analog Focal Plane Sensor Processors
• Automated Backend-Aware Post-Training Quantization
• RA-BNN: Constructing Robust & Accurate Binary Neural Network to Simultaneously Defend Adversarial Bit-Flip Attack and Improve Accuracy
• n-hot: Efficient bit-level sparsity for powers-of-two neural network quantization
• Network Automatic Pruning: Start NAP and Take a Nap
• One Timestep is All You Need: Training Spiking Neural Networks with Ultra Low Latency
• Post-Training Quantization for Vision Transformer
• Histogram-Equalized Quantization for logic-gated Residual Neural Networks
• Distribution-sensitive Information Retention for Accurate Binary Neural Network
• S2-BNN: Bridging the Gap Between Self-Supervised Real and 1-bit Neural Networks via Guided Distribution Calibration
• Reducing the Computational Cost of Deep Generative Models with Binary Neural Networks
• High-Capacity Expert Binary Networks
• Path Sample-Analytic Gradient Estimators for Stochastic Binary Networks
• Learning Sparse & Ternary Neural Networks with Entropy-Constrained Trained Ternarization (EC2T)
• Joint Architecture and Knowledge Distillation in CNN for Chinese Text Recognition
• Active Subspace of Neural Networks: Structural Analysis and Universal Attacks
• MemNet: Memory-Efficiency Guided Neural Architecture Search with Augment-Trim learning
• Towards Fast and Energy-Efficient Binarized Neural Network Inference on FPGA
• In-memory multiplication engine with SOT-MRAM based stochastic computing
• EasyConvPooling: Random Pooling with Easy Convolution for Accelerating Training and Testing
• Merging and Evolution: Improving Convolutional Neural Networks for Mobile Applications
• Low complexity convolutional neural network for vessel segmentation in portable retinal diagnostic devices
• AutoScale: Optimizing Energy Efficiency of End-to-End Edge Inference under Stochastic Variance
• ThumbNet: One Thumbnail Image Contains All You Need for Recognition
• Balanced Binary Neural Networks with Gated Residual
• HAO: Hardware-aware neural Architecture Optimization for Efficient Inference
• Content-Aware GAN Compression
• BCNN: Binary Complex Neural Network
• Improved robustness of reinforcement learning policies upon conversion to spiking neuronal network platforms applied to ATARI games
• Prototype-based Personalized Pruning
• SiMaN: Sign-to-Magnitude Network Binarization
• PartsNet: A Unified Deep Network for Automotive Engine Precision Parts Defect Detection
• Constrained Differentially Private Federated Learning for Low-bandwidth Devices
• Processing-In-Memory Acceleration of Convolutional Neural Networks for Energy-Efficiency, and Power-Intermittency Resilience
• QuPeL: Quantized Personalization with Applications to Federated Learning
• BNAS v2: Learning Architectures for Binary Networks with Empirical Improvements
• PR-CIM: a Variation-Aware Binary-Neural-Network Framework for Process-Resilient Computation-in-memory
• Improving Efficiency in Neural Network Accelerator Using Operands Hamming Distance optimization
• RTN: Reparameterized Ternary Network
• A Machine Learning Accelerator In-Memory for Energy Harvesting
• Binary-decomposed DCNN for accelerating computation and compressing model without retraining
• Mining Mid-level Visual Patterns with Deep CNN Activations
• Transparent Classification with Multilayer Logical Perceptrons and Random Binarization
• A Very Compact Embedded CNN Processor Design Based on Logarithmic Computing
• Ghost Units Yield Biologically Plausible Backprop in Deep Neural Networks
• Learning Accurate Decision Trees with Bandit Feedback via Quantized Gradient Descent
• GDRQ: Group-based Distribution Reshaping for Quantization
• Localization-aware Channel Pruning for Object Detection
• BitHEP -- The Limits of Low-Precision ML in HEP
• AQD: Towards Accurate Fully-Quantized Object Detection
• Training of Quantized Deep Neural Networks using a Magnetic Tunnel Junction-Based Synapse
• Hardware-friendly Neural Network Architecture for Neuromorphic Computing
• ReCU: Reviving the Dead Weights in Binary Neural Networks
• Inherent Weight Normalization in Stochastic Neural Networks
• Incomplete Dot Products for Dynamic Computation Scaling in Neural Network Inference
• ENSEI: Efficient Secure Inference via Frequency-Domain Homomorphic Convolution for Privacy-Preserving Visual Recognition
• Factors for the Generalisation of Identity Relations by Neural Networks
• Exploration of Numerical Precision in Deep Neural Networks
• Running Neural Networks on the NIC
• Defending against substitute model black box adversarial attacks with the 01 loss
• Exploration of Hardware Acceleration Methods for an XNOR Traffic Signs Classifier
• Recent Advances in Convolutional Neural Network Acceleration
• Understanding Learning Dynamics of Binary Neural Networks via Information Bottleneck
• Quantum Annealing Formulation for Binary Neural Networks
• Group Pruning using a Bounded-Lp norm for Group Gating and Regularization
• Enabling Binary Neural Network Training on the Edge
• Fast Walsh-Hadamard Transform and Smooth-Thresholding Based Binary Layers in Deep Neural Networks
• PSDNet and DPDNet: Efficient channel expansion, Depthwise-Pointwise-Depthwise Inverted Bottleneck Block
• Arch-Net: Model Distillation for Architecture Agnostic Model Deployment
• Gator: Customizable Channel Pruning of Neural Networks with Gating
• Towards thinner convolutional neural networks through Gradually Global Pruning
• Is In-Domain Data Really Needed? A Pilot Study on Cross-Domain Calibration for Network Quantization
• An Inter-Layer Weight Prediction and Quantization for Deep Neural Networks based on a Smoothly Varying Weight Hypothesis
• An In-Memory Analog Computing Co-Processor for Energy-Efficient CNN Inference on Mobile Devices
• Efficient Micro-Structured Weight Unification and Pruning for Neural Network Compression
• Efficient Inference via Universal LSH Kernel
• FLASH: Fast Neural Architecture Search with Hardware Optimization
• Complexity-aware Adaptive Training and Inference for Edge-Cloud Distributed AI Systems
• Neural Rejuvenation: Improving Deep Network Training by Enhancing Computational Resource Utilization
• Reducing Inference Latency with Concurrent Architectures for Image Recognition
• DiffPrune: Neural Network Pruning with Deterministic Approximate Binary Gates and $L_0$ Regularization
• BiSNN: Training Spiking Neural Networks with Binary Weights via Bayesian Learning
• Training DNNs in O(1) memory with MEM-DFA using Random Matrices
• "BNN - BN = ?": Training Binary Neural Networks without Batch Normalization
• Entropy-Based Modeling for Estimating Soft Errors Impact on Binarized Neural Network Inference
• End-to-end Learned Image Compression with Fixed Point Weight Quantization
• Robust Student Network Learning
• On the transferability of adversarial examples between convex and 01 loss models
• Scaling Neural Network Performance through Customized Hardware Architectures on Reconfigurable Logic
• Understanding the Energy and Precision Requirements for Online Learning
• Disentangling Neural Architectures and Weights: A Case Study in Supervised Classification
• SoFAr: Shortcut-based Fractal Architectures for Binary Convolutional Neural Networks
• Minimizing Area and Energy of Deep Learning Hardware Design Using Collective Low Precision and Structured Compression
• EDEN: Enabling Energy-Efficient, High-Performance Deep Neural Network Inference Using Approximate DRAM
• Quantization Mimic: Towards Very Tiny CNN for Object Detection
• Memory Requirement Reduction of Deep Neural Networks Using Low-bit Quantization of Parameters
• Classification Accuracy Improvement for Neuromorphic Computing Systems with One-level Precision Synapses
• FeTa: A DCA Pruning Algorithm with Generalization Error Guarantees
• Training Modern Deep Neural Networks for Memory-Fault Robustness
• Recurrent Binary Embedding for GPU-Enabled Exhaustive Retrieval from Billion-Scale Semantic Vectors
• PhoneBit: Efficient GPU-Accelerated Binary Neural Network Inference Engine for Mobile Phones
• Learning to Request Guidance in Emergent Communication
• Widening and Squeezing: Towards Accurate and Efficient QNNs
• PBGen: Partial Binarization of Deconvolution-Based Generators for Edge Intelligence
• Balancing Cost and Benefit with Tied-Multi Transformers
• A Main/Subsidiary Network Framework for Simplifying Binary Neural Network
• Studying the Plasticity in Deep Convolutional Neural Networks using Random Pruning
• Dynamic Multi-path Neural Network
• Perturbative GAN: GAN with Perturbation Layers
• Revisiting hard thresholding for DNN pruning
• Deep Learning Acceleration Techniques for Real Time Mobile Vision Applications
• Target Curricula via Selection of Minimum Feature Sets: a Case Study in Boolean Networks
• Structured Sparsification with Joint Optimization of Group Convolution and Channel Shuffle
• Towards Understanding Residual and Dilated Dense Neural Networks via Convolutional Sparse Coding
• Hardware and software co-optimization for the initialization failure of the ReRAM based cross-bar array
• Accuracy to Throughput Trade-offs for Reduced Precision Neural Networks on Reconfigurable Logic
• Modulated binary cliquenet
• Auto Deep Compression by Reinforcement Learning Based Actor-Critic Structure
• Linear Delay-cell Design for Low-energy Delay Multiplication and Accumulation
• Cluster Regularized Quantization for Deep Networks Compression
• Towards Modality Transferable Visual Information Representation with Optimal Model Compression
• Towards adversarial robustness with 01 loss neural networks
• ViP: Virtual Pooling for Accelerating CNN-based Image Classification and Object Detection
• Recurrent Stacking of Layers in Neural Networks: An Application to Neural Machine Translation
• On the Quantization of Cellular Neural Networks for Cyber-Physical Systems
• BitSplit-Net: Multi-bit Deep Neural Network with Bitwise Activation Function
• Outstanding Bit Error Tolerance of Resistive RAM-Based Binarized Neural Networks
• Compressing Facial Makeup Transfer Networks by Collaborative Distillation and Kernel Decomposition
• Improving DNN Fault Tolerance using Weight Pruning and Differential Crossbar Mapping for ReRAM-based Edge AI
• Distilling Image Classifiers in Object Detectors
• Full-stack Optimization for Accelerating CNNs with FPGA Validation
• Recent Advances in Large Margin Learning
• Self-Adaptive Network Pruning
• Circulant Binary Convolutional Networks: Enhancing the Performance of 1-bit DCNNs with Circulant Back Propagation
• DTNN: Energy-efficient Inference with Dendrite Tree Inspired Neural Networks for Edge Vision Applications
• Seeing Convolution Through the Eyes of Finite Transformation Semigroup Theory: An Abstract Algebraic Interpretation of Convolutional Neural Networks
• An Open-Source Framework for Efficient Numerically-Tailored Computations
• A Computing Kernel for Network Binarization on PyTorch
• Online Filter Clustering and Pruning for Efficient Convnets
• A Highly Effective Low-Rank Compression of Deep Neural Networks with Modified Beam-Search and Modified Stable Rank
• Quantization Loss Re-Learning Method
• Random Bias Initialization Improves Quantized Training
• Radius Adaptive Convolutional Neural Network
• Recurrent Residual Module for Fast Inference in Videos
• Low-bit Quantization of Recurrent Neural Network Language Models Using Alternating Direction Methods of Multipliers
• Bayesian Optimized 1-Bit CNNs
• Fixed-point Factorized Networks
• Stochastic Computing for Hardware Implementation of Binarized Neural Networks
• ClosNets: a Priori Sparse Topologies for Faster DNN Training
• Be Your Own Best Competitor! Multi-Branched Adversarial Knowledge Transfer
• MoRS: An Approximate Fault Modelling Framework for Reduced-Voltage SRAMs
• OICSR: Out-In-Channel Sparsity Regularization for Compact Deep Neural Networks
• A Targeted Acceleration and Compression Framework for Low bit Neural Networks
• Binarized Weight Error Networks With a Transition Regularization Term
• Hardware-Software Codesign of Accurate, Multiplier-free Deep Neural Networks
• PoET-BiN: Power Efficient Tiny Binary Neurons
• Class-Discriminative CNN Compression
• Efficient and Robust Mixed-Integer Optimization Methods for Training Binarized Deep Neural Networks
• CBP: Backpropagation with constraint on weight precision using a pseudo-Lagrange multiplier method
• RMSMP: A Novel Deep Neural Network Quantization Framework with Row-wise Mixed Schemes and Multiple Precisions
• HOTCAKE: Higher Order Tucker Articulated Kernels for Deeper CNN Compression
• A Note on Latency Variability of Deep Neural Networks for Mobile Inference
• Binary Latent Representations for Efficient Ranking: Empirical Assessment
• Learning Quantized Neural Nets by Coarse Gradient Method for Non-linear Classification
• A Low Power In-Memory Multiplication andAccumulation Array with Modified Radix-4 Inputand Canonical Signed Digit Weights
• NeuroView: Explainable Deep Network Decision Making
• Tetris: Re-architecting Convolutional Neural Network Computation for Machine Learning Accelerators
• Meta-Aggregator: Learning to Aggregate for 1-bit Graph Neural Networks
• Architecture Aware Latency Constrained Sparse Neural Networks
• Identifying and Exploiting Structures for Reliable Deep Learning
• Penetrating the Fog: the Path to Efficient CNN Models
• Homomorphic Parameter Compression for Distributed Deep Learning Training
• Binarized Neural Networks for Resource-Constrained On-Device Gait Identification
• Efficient Hardware Realizations of Feedforward Artificial Neural Networks
• ENOS: Energy-Aware Network Operator Search for Hybrid Digital and Compute-in-Memory DNN Accelerators
• cvpaper.challenge in 2016: Futuristic Computer Vision through 1,600 Papers Survey
• Formation of cell assemblies with iterative winners-take-all computation and excitation-inhibition balance
• MICIK: MIning Cross-Layer Inherent Similarity Knowledge for Deep Model Compression
• Automatic low-bit hybrid quantization of neural networks through meta learning
• Low Power In-Memory Implementation of Ternary Neural Networks with Resistive RAM-Based Synapse
• Transfer Learning with Binary Neural Networks
• Differentiable Architecture Pruning for Transfer Learning
• CP-NAS: Child-Parent Neural Architecture Search for Binary Neural Networks
• Cross-filter compression for CNN inference acceleration
• An Overview of Datatype Quantization Techniques for Convolutional Neural Networks
• In-memory Implementation of On-chip Trainable and Scalable ANN for AI/ML Applications
• Neural Network Activation Quantization with Bitwise Information Bottlenecks
• Dep-$L_0$: Improving $L_0$-based Network Sparsification via Dependency Modeling
• ABC-Di: Approximate Bayesian Computation for Discrete Data
• Convergence of a Relaxed Variable Splitting Coarse Gradient Descent Method for Learning Sparse Weight Binarized Activation Neural Networks
• NN2CAM: Automated Neural Network Mapping for Multi-Precision Edge Processing on FPGA-Based Cameras
• Embracing the Unreliability of Memory Devices for Neuromorphic Computing