Regularize or Localize: When Training-Time KV-Cache Geometry Pays Under Quantization

Libo Sun, Po-Wei Harn, Zewei Zhang +2

We study whether \sigreg -- LeJEPA's anti-collapse objective -- can reshape representations during standard autoregressive language-model pretraining, and when the resulting geomet…