71 citations · 92 across the 8 of their papers we have counts for
5 papers · 1 filter
Learning to Remember, Forget and Ignore using Attention Control in Memory
T. S. Jayram, Younes Bouhadjar, Ryan L. McAvoy +4
Typical neural networks with external memory do not effectively separate capacity for episodic and working memory as is required for reasoning in humans. Applying knowledge gained…
Surprisal-Driven Zoneout
Kamil Rocki, Tomasz Kornuta, Tegan Maharaj
We propose a novel method of regularization for recurrent neural networks called suprisal-driven zoneout. In this method, states zoneout (maintain their previous value rather than…
Surprisal-Driven Feedback in Recurrent Networks
Kamil M Rocki
Recurrent neural nets are widely used for predicting temporal data. Their inherent deep feedforward structure allows learning complex sequential patterns. It is believed that top-d…
Recurrent Memory Array Structures
Kamil Rocki
The following report introduces ideas augmenting standard Long Short Term Memory (LSTM) architecture with multiple memory cells per hidden unit in order to improve its generalizati…
Thinking Required
Kamil Rocki
There exists a theory of a single general-purpose learning algorithm which could explain the principles its operation. It assumes the initial rough architecture, a small library of…