4 papers
On the Expressiveness of Softmax Attention: A Recurrent Neural Network Perspective
Gabriel Mongaras, Eric C. Larson
Since its introduction, softmax attention has become the backbone of modern transformer architectures due to its expressiveness and scalability across a wide range of tasks. Howeve…
Equitable Electronic Health Record Prediction with FAME: Fairness-Aware Multimodal Embedding
Nikkie Hooman, Zhongjie Wu, Eric C. Larson +1
Electronic Health Record (EHR) data encompass diverse modalities -- text, images, and medical codes -- that are vital for clinical decision-making. To process these complex data, m…
Cottention: Linear Transformers With Cosine Attention
Gabriel Mongaras, Trevor Dohm, Eric C. Larson
Attention mechanisms, particularly softmax attention, have been instrumental in the success of transformer-based models such as GPT. However, the quadratic memory complexity of sof…
Digit Recognition using Multimodal Spiking Neural Networks
William Bjorndahl, Jack Easton, Austin Modoff +3
Spiking neural networks (SNNs) are the third generation of neural networks that are biologically inspired to process data in a fashion that emulates the exchange of signals in the…