3 papers
cs.LG2026
Predictive Objectives Discard Exogenous Control-Relevant Features: A Controlled Mechanistic Study
Ayan Pendharkar
Joint-embedding predictive (JEPA-style) objectives learn representations by predicting future latents. In doing so they can discard features that are exogenous (uncontrollable by t…
cs.LG2026
Structural Grid Descriptors Predict Within-Task Solver Success on ARC-AGI
Ayan Pendharkar
We ask whether structural properties of intermediate grid states predict whether a symbolic ARC-AGI solver will succeed, framed as a test of conditional mutual information I(X;Y|ta…
cs.LG2026
Gradient Flow Structure and Quantitative Dynamics of Multi-Head Self-Attention
Ayan Pendharkar
Transformer self-attention can be interpreted as a gradient flow on the unit sphere, in which tokens evolve under softmax interaction potentials and tend to form clusters. While pr…