TextNCA: Neural Cellular Automata for Language Modeling via Hierarchical Local Attention

Avni Mittal, Avinash Anand, Ashutosh Kumar +7

Can a strictly local, iterated, weight-shared computation primitive support language modelling, and which of those three properties actually drives the model's behaviour? We define…