5 papers
Winner-Take-All Spiking Transformer for Language Modeling
Chenlin Zhou, Sihang Guo, Jiaqi Wang +6
Spiking Transformers, which combine the scalability of Transformers with the sparse, energy-efficient property of Spiking Neural Networks (SNNs), have achieved impressive results i…
BrainFuse: a unified infrastructure integrating realistic biological modeling and core AI methodology
Baiyu Chen, Yujie Wu, Siyuan Xu +9
Neuroscience and artificial intelligence represent distinct yet complementary pathways to general intelligence. However, amid the ongoing boom in AI research and applications, the…
Temporal-adaptive Weight Quantization for Spiking Neural Networks
Han Zhang, Qingyan Meng, Jiaqi Wang +3
Weight quantization in spiking neural networks (SNNs) could further reduce energy consumption. However, quantizing weights without sacrificing accuracy remains challenging. In this…
Optimal Dynamic Regret by Transformers for Non-Stationary Reinforcement Learning
Baiyuan Chen, Shinji Ito, Masaaki Imaizumi
Transformers have demonstrated exceptional performance across a wide range of domains. While their ability to perform reinforcement learning in-context has been established both th…
Is Smoothness the Key to Robustness? A Comparison of Attention and Convolution Models Using a Novel Metric
Baiyuan Chen
Robustness is a critical aspect of machine learning models. Existing robustness evaluation approaches often lack theoretical generality or rely heavily on empirical assessments, li…