Isotropic Yet Undecodable: The Sequential Content-Sufficiency Gap in Latent-Predictive Text Representations

K. P. Santoso, N. Z. Fadil, F. P. Harsanti +2

We study sequential content sufficiency by investigating whether a representation retains the ordered target information available in its input. An information-theoretic decomposit…