SpikeOPD: Stable On-Policy Distillation for Autoregressive Spiking Language Models

Enqiao Lu, Xingrui Yu, Yiwei Fu +7

Spiking neural networks (SNNs) offer a path to energy-efficient language modeling through sparse encoding and event-driven computation, but training capable spiking language models…