Fundamental Limitations in Sequential Prediction and Recursive Algorithms: ${\mathcal{L}}_p$ Bounds via an Entropic Analysis

Abstract

In this paper, we obtain fundamental ${\mathcal{L}}_p$ bounds in sequential prediction and recursive algorithms via an entropic analysis. Both classes of problems are examined by investigating the underlying entropic relationships of the data and/or noises involved, and the derived lower bounds may all be quantified in a conditional entropy characterization. We also study the conditions to achieve the generic bounds from an innovations' viewpoint.

arXiv admin note: substantial text overlap with arXiv:1910.06742. text overlap with arXiv:1912.05541

References in corpus (2)

• Opening the Black Box of Deep Neural Networks via Information
• Generic Variance Bounds on Estimation and Prediction Errors in Time Series Analysis: An Entropy Perspective