Test-Time Training Done Right

Tianyuan Zhang, Sai Bi, Yicong Hong +6

Test-Time Training (TTT) models context dependencies by adapting part of the model's weights (referred to as fast weights) during inference. This fast weight, akin to recurrent sta…