Do Smaller Language Models Answer Contextualised Questions Through Memorisation Or Generalisation?

Tim Hartill, Joshua Bensemann, Michael Witbrock +1

A distinction is often drawn between a model's ability to predict a label for an evaluation sample that is directly memorised from highly similar training samples versus an ability…