The Illusion of Replacement: Rethinking Specialized Machine Learning Models in the Foundation Model Era

Kiyan Rezaee

Specialized machine learning architectures encode structural assumptions (equivariance, permutation invariance, relational structure) that language-based foundation models lack by…