Matched Specialization, Different Adaptation: Second-Moment Structure in Shared Representations

Abstract

Small Adam-trained shared-representation networks can reach equivalent selected mean readout measurements yet respond differently to the same further learning. We use state interventions to separate second-moment scale and coordinate-pattern effects, then examine their formation and time dependence. In the central two-target experiment, the more precisely readable target switches within ten updates in 67.5% of acquisition-formed checkpoints and 0% of update-formed checkpoints. First-moment exchange leaves the ten-update contrast change within its pre-specified margin, whereas second-moment exchange reverses the contrast. A coordinate-pattern effect remains at common mean adaptive scale. Layer and movement controls locate an effect beyond overall update size. Formation records connect small encoder second moments with stronger cancellation between task gradients. A prospective gradient-projection experiment changes this retained state; transferring the resulting second moment alone reduces the ten-update adaptation response in a fixed recipient. The matched comparison extends to three Gaussian targets, while a room-sensor extension retains component effects but fails endpoint matching. Across tested regimes, component effects survive more widely than reversal or reset equivalence. Effect magnitude and duration depend on learning conditions.

arXiv comment18 pages including appendices; 5 figures; 8 tables. Retitled and substantially revised, with new formation-mechanism experiments, layer and movement controls, and three-target adaptation and sensor-data extensions. Code and data: https://github.com/culturegeek/matched-specialization-different-adaptation