Neural Information Causality: A Random-Access Capacity Law for Representation Learning

Abstract

How can a learned representation remain useful when the question it must answer is chosen only after encoding? Preserving information is essential, but does not by itself ensure access to the requested target. We address this problem by deriving "neural information causality (Neural-IC)," a representation law that connects causal access to the physically specified interface capacity. For a fresh database of independent bits, the aggregate information recoverable across later queries cannot exceed that capacity, even with pre-established quantum resources. A decomposition separates capacity slack, information organized jointly across targets, and losses in producing answers. Exhaustive evaluation of trained binary codes reveals why nearly full use of a memory's information budget can coexist with limited retrieval. In a nested Bell-correlation construction, quantum assistance improves average retrieval over the optimal classical one-bit benchmark within the same communicated budget. These results give representation learning a design principle that connects information capacity to the organization and extraction of target information. By locating the constraints on later answers, the framework can inform memory allocation, encoding objectives, and decoder outputs, providing a basis for seeking better retrieval within a fixed budget.

arXiv comment13 pages, 3 figures (including Appendix)