Statistical mechanics of lossy data compression using a non-monotonic perceptron

Abstract

The performance of a lossy data compression scheme for uniformly biased Boolean messages is investigated via methods of statistical mechanics. Inspired by a formal similarity to the storage capacity problem in the research of neural networks, we utilize a perceptron of which the transfer function is appropriately designed in order to compress and decode the messages. Employing the replica method, we analytically show that our scheme can achieve the optimal performance known in the framework of lossy compression in most cases when the code length becomes infinity. The validity of the obtained results is numerically confirmed.

9 pages, 5 figures, Physical Review E

Cited by in corpus (20)

• Optimal Errors and Phase Transitions in High-Dimensional Generalized Linear Models
• Analysis of CDMA systems that are characterized by eigenvalue spectrum
• Origin of the computational hardness for learning with binary synapses
• Lossy data compression with random gates
• Storage capacity in symmetric binary perceptrons
• Perceptron capacity revisited: classification ability for correlated patterns
• One step RSB scheme for the rate distortion function
• Statistical mechanics of lossy compression using multilayer perceptrons
• Learning of correlated patterns by simple perceptrons
• The theoretical capacity of the Parity Source Coder
• Statistical Mechanical Approach to Lossy Data Compression:Theory and Practice
• Statistical mechanics of lossy compression for non-monotonic multilayer perceptrons
• Understanding the computational difficulty of a binary-weight perceptron and the advantage of input sparseness
• Replication-based Inference Algorithms for Hard Computational Problems
• Linear Complexity Lossy Compressor for Binary Redundant Memoryless Sources
• Belief Propagation for Error Correcting Codes and Lossy Compression Using Multilayer Perceptrons
• Universal Behavior in Large-scale Aggregation of Independent Noisy Observations
• Parallel dynamics of continuous Hopfield model revisited
• Error correcting code using tree-like multilayer perceptron
• Statistical mechanical analysis of a hierarchical random code ensemble in signal processing