Statistical mechanics of lossy compression for non-monotonic multilayer perceptrons
arXiv:0807.4009 · doi:10.1103/PhysRevE.78.021124
Abstract
A lossy data compression scheme for uniformly biased Boolean messages is investigated via statistical mechanics techniques. We utilize tree-like committee machine (committee tree) and tree-like parity machine (parity tree) whose transfer functions are non-monotonic. The scheme performance at the infinite code length limit is analyzed using the replica method. Both committee and parity treelike networks are shown to saturate the Shannon bound. The AT stability of the Replica Symmetric solution is analyzed, and the tuning of the non-monotonic transfer function is also discussed.
29 pages, 7 figures
References in corpus (8)
- Statistical mechanics of image restoration and error-correcting codes
- Typical Performance of Gallager-type Error-Correcting Codes
- The statistical mechanics of turbo codes
- Thouless-Anderson-Palmer Approach for Lossy Compression
- Statistical mechanics of lossy data compression using a non-monotonic perceptron
- One step RSB scheme for the rate distortion function
- Statistical mechanics of lossy compression using multilayer perceptrons
- Statistical Mechanical Approach to Lossy Data Compression:Theory and Practice
Cited by in corpus (7)
- Optimum estimation via gradients of partition functions and information measures: a statistical-mechanical perspective
- Linear Complexity Lossy Compressor for Binary Redundant Memoryless Sources
- Parallel dynamics of continuous Hopfield model revisited
- Belief Propagation for Error Correcting Codes and Lossy Compression Using Multilayer Perceptrons
- Error correcting code using tree-like multilayer perceptron
- A statistical-mechanical view on source coding: physical compression and data compression
- Analysis of Mismatched Estimation Errors Using Gradients of Partition Functions