3 papers
cs.NE2024
Towards 3D Acceleration for low-power Mixture-of-Experts and Multi-Head Attention Spiking Transformers
Boxun Xu, Junyoung Hwang, Pruek Vanna-iampikul +3
Spiking Neural Networks(SNNs) provide a brain-inspired and event-driven mechanism that is believed to be critical to unlock energy-efficient deep learning. The mixture-of-experts a…
cs.NE2024
Spiking Transformer Hardware Accelerators in 3D Integration
Boxun Xu, Junyoung Hwang, Pruek Vanna-iampikul +2
Spiking neural networks (SNNs) are powerful models of spatiotemporal computation and are well suited for deployment on resource-constrained edge devices and neuromorphic hardware d…
cs.AI2024
Federating to Grow Transformers with Constrained Resources without Model Sharing
Shikun Shen, Yifei Zou, Yuan Yuan +4
The high resource consumption of large-scale models discourages resource-constrained users from developing their customized transformers. To this end, this paper considers a federa…