A Systematic Study of Small Language Models on Abstract Reasoning Tasks

Nur A Zarin Nishat, Jens Lehmann, Andrei Aioanei +1

Endpoint accuracy on abstract-reasoning benchmarks does not reveal whether a language model has acquired a transferable rule or fit distribution-specific regularities. We study thi…