4 papers · 1 filter
Event-triggered Implicit Perturbation for Zeroth-Order Fine-Tuning of Spiking Transformers
Tengteng Lei, Prabodh Katti, Rashi Dutt +5
Zeroth-order (ZO) optimization estimates gradients using only forward-pass evaluations, making it suitable for fine-tuning non-differentiable, event-driven spiking neural networks…
Efficient transformer adaptation for analog in-memory computing via low-rank adapters
Chen Li, Elena Ferro, Corey Lammie +3
Analog In-Memory Computing (AIMC) offers a promising solution to the von Neumann bottleneck. However, deploying transformer models on AIMC remains challenging due to their inherent…
Xpikeformer: Hybrid Analog-Digital Hardware Acceleration for Spiking Transformers
Zihang Song, Prabodh Katti, Osvaldo Simeone +1
The integration of neuromorphic computing and transformers through spiking neural networks (SNNs) offers a promising path to energy-efficient sequence modeling, with the potential…
Stochastic Spiking Attention: Accelerating Attention with Stochastic Computing in Spiking Networks
Zihang Song, Prabodh Katti, Osvaldo Simeone +1
Spiking Neural Networks (SNNs) have been recently integrated into Transformer architectures due to their potential to reduce computational demands and to improve power efficiency.…