On Effects of Compression with Hyperdimensional Computing in Distributed Randomized Neural Networks

Abstract

A change of the prevalent supervised learning techniques is foreseeable in the near future: from the complex, computational expensive algorithms to more flexible and elementary training ones. The strong revitalization of randomized algorithms can be framed in this prospect steering. We recently proposed a model for distributed classification based on randomized neural networks and hyperdimensional computing, which takes into account cost of information exchange between agents using compression. The use of compression is important as it addresses the issues related to the communication bottleneck, however, the original approach is rigid in the way the compression is used. Therefore, in this work, we propose a more flexible approach to compression and compare it to conventional compression algorithms, dimensionality reduction, and quantization techniques.

arXiv comment12 pages, 3 figures

References in corpus (6)

• Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations
• Density Encoding Enables Resource-Efficient Randomly Connected Neural Networks
• Superposition of many models into one
• Autoscaling Bloom Filter: Controlling Trade-off Between True and False Positives
• HyperEmbed: Tradeoffs Between Resources and Performance in NLP Tasks with Hyperdimensional Computing enabled Embedding of n-gram Statistics
• Perceptron Theory Can Predict the Accuracy of Neural Networks

Cited by in corpus (1)

• A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part II: Applications, Cognitive Models, and Challenges