Information Locality as an Inductive Bias for Neural Language Models

Taiga Someya, Anej Svete, Brian DuSell +3

Inductive biases are inherent in every machine learning system, shaping how models generalize from finite data. In the case of neural language models (LMs), debates persist as to w…