6 papers
Binary Event-Driven Spiking Transformer
Honglin Cao, Zijian Zhou, Wenjie Wei +6
Transformer-based Spiking Neural Networks (SNNs) introduce a novel event-driven self-attention paradigm that combines the high performance of Transformers with the energy efficienc…
Unveiling the Spatial-temporal Effective Receptive Fields of Spiking Neural Networks
Jieyuan Zhang, Xiaolong Zhou, Shuai Wang +6
Spiking Neural Networks (SNNs) demonstrate significant potential for energy-efficient neuromorphic computing through an event-driven paradigm. While training methods and computatio…
SNN: Sub-bit Spiking Neural Networks
Wenjie Wei, Malu Zhang, Jieyuan Zhang +8
Spiking Neural Networks (SNNs) offer an energy-efficient paradigm for machine intelligence, but their continued scaling poses challenges for resource-limited deployment. Despite re…
Dendritic Resonate-and-Fire Neuron for Effective and Efficient Long Sequence Modeling
Dehao Zhang, Malu Zhang, Shuai Wang +6
The explosive growth in sequence length has intensified the demand for effective and efficient long sequence modeling. Benefiting from intrinsic oscillatory membrane dynamics, Reso…
What Is Next for LLMs? Next-Generation AI Computing Hardware Using Photonic Chips
Renjie Li, Wenjie Wei, Qi Xin +7
Large language models (LLMs) are rapidly pushing the limits of contemporary computing hardware. For example, training GPT-3 has been estimated to consume around 1300 MWh of electri…
Quantized Spike-driven Transformer
Xuerui Qiu, Malu Zhang, Jieyuan Zhang +7
Spiking neural networks are emerging as a promising energy-efficient alternative to traditional artificial neural networks due to their spike-driven paradigm. However, recent resea…