Showing cs.LGShow all
3 papers · 1 filter
cs.LG2025
Emergence of Episodic Memory in Transformers: Characterizing Changes in Temporal Structure of Attention Scores During Training
Deven Mahesh Mistry, Anooshka Bajaj, Yash Aggarwal +2
We investigate in-context temporal biases in attention heads and transformer outputs. Using cognitive science methodologies, we analyze attention scores and outputs of the GPT-2 mo…
cs.LG2021
A deep convolutional neural network that is invariant to time rescaling
Brandon G. Jacques, Zoran Tiganj, Aakash Sarkar +2
Human learners can readily understand speech, or a melody, when it is presented slower or faster than usual. Although deep convolutional neural networks (CNNs) are extremely powerf…
cs.LG2021
DeepSITH: Efficient Learning via Decomposition of What and When Across Time Scales
Brandon Jacques, Zoran Tiganj, Marc W. Howard +1
Extracting temporal relationships over a range of scales is a hallmark of human perception and cognition -- and thus it is a critical feature of machine learning applied to real-wo…