Wiola 13M, a Gated Spiral Attention Architecture for Parameter Efficient Small Language Models

Aryuemaan Kumar Chowdhury, Praveen Oosa, Vineesha Reddy

Small language models in the ten to one hundred million parameter range are attractive for on device inference, rapid experimentation, and controlled scientific study, yet most of…