Independence Tests for Language Models

Sally Zhu, Ahmed Ahmed, Rohith Kuditipudi +1

We consider the following problem: given the weights of two models, can we test whether they were trained independently -- i.e., from independent random initializations? We conside…