11 papers
BiSpikCLM: A Spiking Language Model integrating Softmax-Free Spiking Attention and Spike-Aware Alignment Distillation
Sihang Guo, Chenlin Zhou, Jiaqi Wang +3
Spiking Neural Networks (SNNs) offer promising energy-efficient alternatives to large language models (LLMs) due to their event-driven nature and ultra-low power consumption. Howev…
Adaptive Spiking Neurons for Vision and Language Modeling
Chenlin Zhou, Sihang Guo, Jiaqi Wang +5
Regarded as the third generation of neural networks, Spiking Neural Networks (SNNs) have garnered significant traction due to their biological plausibility and energy efficiency. R…
Winner-Take-All Spiking Transformer for Language Modeling
Chenlin Zhou, Sihang Guo, Jiaqi Wang +6
Spiking Transformers, which combine the scalability of Transformers with the sparse, energy-efficient property of Spiking Neural Networks (SNNs), have achieved impressive results i…
SpikCommander: A High-performance Spiking Transformer with Multi-view Learning for Efficient Speech Command Recognition
Jiaqi Wang, Liutao Yu, Xiongri Shen +6
Spiking neural networks (SNNs) offer a promising path toward energy-efficient speech command recognition (SCR) by leveraging their event-driven processing paradigm. However, existi…
Spikingformer: A Key Foundation Model for Spiking Neural Networks
Chenlin Zhou, Liutao Yu, Zhaokun Zhou +5
Spiking neural networks (SNNs) offer a promising energy-efficient alternative to artificial neural networks, due to their event-driven spiking computation. However, some foundation…
SM-Former: Spiking Symmetric Mixing Branchformer for Brain Auditory Attention Detection
Jiaqi Wang, Zhengyu Ma, Xiongri Shen +7
Auditory attention detection (AAD) aims to decode listeners' focus in complex auditory environments from electroencephalography (EEG) recordings, which is crucial for developing ne…