DFTerNet: Towards 2-bit Dynamic Fusion Networks for Accurate Human Activity Recognition

Abstract

Deep Convolutional Neural Networks (DCNNs) are currently popular in human activity recognition applications. However, in the face of modern artificial intelligence sensor-based games, many research achievements cannot be practically applied on portable devices. DCNNs are typically resource-intensive and too large to be deployed on portable devices, thus this limits the practical application of complex activity detection. In addition, since portable devices do not possess high-performance Graphic Processing Units (GPUs), there is hardly any improvement in Action Game (ACT) experience. Besides, in order to deal with multi-sensor collaboration, all previous human activity recognition models typically treated the representations from different sensor signal sources equally. However, distinct types of activities should adopt different fusion strategies. In this paper, a novel scheme is proposed. This scheme is used to train 2-bit Convolutional Neural Networks with weights and activations constrained to {-0.5,0,0.5}. It takes into account the correlation between different sensor signal sources and the activity types. This model, which we refer to as DFTerNet, aims at producing a more reliable inference and better trade-offs for practical applications. Our basic idea is to exploit quantization of weights and activations directly in pre-trained filter banks and adopt dynamic fusion strategies for different activity types. Experiments demonstrate that by using dynamic fusion strategy can exceed the baseline model performance by up to ~5% on activity recognition like OPPORTUNITY and PAMAP2 datasets. Using the quantization method proposed, we were able to achieve performances closer to that of full-precision counterpart. These results were also verified using the UniMiB-SHAR dataset. In addition, the proposed method can achieve ~9x acceleration on CPUs and ~11x memory saving.

arXiv comment19 pages, 5 figures, 6 tables, accepted by IEEE Access

References in corpus (21)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• ADADELTA: An Adaptive Learning Rate Method
• Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding
• Fully Convolutional Networks for Semantic Segmentation
• Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
• BinaryConnect: Training Deep Neural Networks with binary weights during propagations
• DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
• Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations
• Deep Learning with Limited Numerical Precision
• QSGD: Communication-Efficient SGD via Gradient Quantization and Encoding
• Convolutional Networks for Fast, Energy-Efficient Neuromorphic Computing
• Ternary Weight Networks
• TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learning
• Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights
• Deep, Convolutional, and Recurrent Models for Human Activity Recognition using Wearables
• Deep Activity Recognition Models with Triaxial Accelerometers
• Estimating or Propagating Gradients Through Stochastic Neurons
• Faster Population Counts Using AVX2 Instructions
• UniMiB SHAR: a new dataset for human activity recognition using acceleration data from smartphones

Cited by in corpus (6)

• Deep Learning for Sensor-based Human Activity Recognition: Overview, Challenges and Opportunities
• Layer-wise training convolutional neural networks with smaller filters for human activity recognition using wearable sensors
• Asymmetric Residual Neural Network for Accurate Human Activity Recognition
• Smartphone Sensor-based Human Activity Recognition Robust to Different Sampling Rates
• Octave Mix: Data augmentation using frequency decomposition for activity recognition
• Memory based fusion for multi-modal deep learning