6 papers · 1 filter
BiSpikCLM: A Spiking Language Model integrating Softmax-Free Spiking Attention and Spike-Aware Alignment Distillation
Sihang Guo, Chenlin Zhou, Jiaqi Wang +3
Spiking Neural Networks (SNNs) offer promising energy-efficient alternatives to large language models (LLMs) due to their event-driven nature and ultra-low power consumption. Howev…
Adaptive Spiking Neurons for Vision and Language Modeling
Chenlin Zhou, Sihang Guo, Jiaqi Wang +5
Regarded as the third generation of neural networks, Spiking Neural Networks (SNNs) have garnered significant traction due to their biological plausibility and energy efficiency. R…
Winner-Take-All Spiking Transformer for Language Modeling
Chenlin Zhou, Sihang Guo, Jiaqi Wang +6
Spiking Transformers, which combine the scalability of Transformers with the sparse, energy-efficient property of Spiking Neural Networks (SNNs), have achieved impressive results i…
Temporal-adaptive Weight Quantization for Spiking Neural Networks
Han Zhang, Qingyan Meng, Jiaqi Wang +3
Weight quantization in spiking neural networks (SNNs) could further reduce energy consumption. However, quantizing weights without sacrificing accuracy remains challenging. In this…
One-Timestep is Enough: Achieving High-performance ANN-to-SNN Conversion via Scale-and-Fire Neurons
Qiuyang Chen, Huiqi Yang, Qingyan Meng +1
Spiking Neural Networks (SNNs) are gaining attention as energy-efficient alternatives to Artificial Neural Networks (ANNs), especially in resource-constrained settings. While ANN-t…
A Self-Ensemble Inspired Approach for Effective Training of Binary-Weight Spiking Neural Networks
Qingyan Meng, Mingqing Xiao, Zhengyu Ma +3
Spiking Neural Networks (SNNs) are a promising approach to low-power applications on neuromorphic hardware due to their energy efficiency. However, training SNNs is challenging bec…