4 papers
Rethinking Attention Locality in Spiking Transformers
Zeqi Zheng, Zizheng Zhu, Yuping Yan +3
Spiking Transformers provide a promising paradigm for efficient visual processing with spike-driven computation, yet their Softmax-free Spiking Self-Attention (SSA) struggles to es…
STF: Shallow-Level Temporal Feedback to Enhance Spiking Transformers
Zeqi Zheng, Zizheng Zhu, Yingchao Yu +5
Transformer-based Spiking Neural Networks (SNNs) suffer from a great performance gap compared to floating-point \mbox{Artificial} Neural Networks (ANNs) due to the binary nature of…
TDFormer: A Top-Down Attention-Controlled Spiking Transformer
Zizheng Zhu, Yingchao Yu, Zeqi Zheng +2
Traditional spiking neural networks (SNNs) can be viewed as a combination of multiple subnetworks with each running for one time step, where the parameters are shared, and the memb…
SpiLiFormer: Enhancing Spiking Transformers with Lateral Inhibition
Zeqi Zheng, Yanchen Huang, Yingchao Yu +4
Spiking Neural Networks (SNNs) based on Transformers have garnered significant attention due to their superior performance and high energy efficiency. However, the spiking attentio…