6 citations · 17 across the 12 of their papers we have counts for
9 papers · 1 filter
RMAAT: Astrocyte-Inspired Memory Compression and Replay for Efficient Long-Context Transformers
Md Zesun Ahmed Mia, Malyaban Bal, Abhronil Sengupta
The quadratic complexity of self-attention mechanism presents a significant impediment to applying Transformer models to long sequences. This work explores computational principles…
Spatio-Temporal Pruning for Compressed Spiking Large Language Models
Yi Jiang, Malyaban Bal, Brian Matejek +3
Large Language Models (LLMs) present significant challenges for deployment in energy-constrained environments due to their large model sizes and high inference latency. Spiking Neu…
Scaling SNNs Trained Using Equilibrium Propagation to Convolutional Architectures
Jiaqi Lin, Malyaban Bal, Abhronil Sengupta
Equilibrium Propagation (EP) is a biologically plausible local learning algorithm initially developed for convergent recurrent neural networks (RNNs), where weight updates rely sol…
Exploring Extreme Quantization in Spiking Language Models
Malyaban Bal, Yi Jiang, Abhronil Sengupta
Despite the growing prevalence of large language model (LLM) architectures, a crucial concern persists regarding their energy and power consumption, which still lags far behind the…
P-SpikeSSM: Harnessing Probabilistic Spiking State Space Models for Long-Range Dependency Tasks
Malyaban Bal, Abhronil Sengupta
Spiking neural networks (SNNs) are posited as a computationally efficient and biologically plausible alternative to conventional neural architectures, with their core computational…
Benchmarking Spiking Neural Network Learning Methods with Varying Locality
Jiaqi Lin, Sen Lu, Malyaban Bal +1
Spiking Neural Networks (SNNs), providing more realistic neuronal dynamics, have been shown to achieve performance comparable to Artificial Neural Networks (ANNs) in several machin…
Delving Deeper Into Astromorphic Transformers
Md Zesun Ahmed Mia, Malyaban Bal, Abhronil Sengupta
Preliminary attempts at incorporating the critical role of astrocytes - cells that constitute more than 50\% of human brain cells - in brain-inspired neuromorphic computing remain…
SpikingBERT: Distilling BERT to Train Spiking Language Models Using Implicit Differentiation
Malyaban Bal, Abhronil Sengupta
Large language Models (LLMs), though growing exceedingly powerful, comprises of orders of magnitude less neurons and synapses than the human brain. However, it requires significant…
Sequence Learning Using Equilibrium Propagation
Malyaban Bal, Abhronil Sengupta
Equilibrium Propagation (EP) is a powerful and more bio-plausible alternative to conventional learning frameworks such as backpropagation. The effectiveness of EP stems from the fa…