5 citations · 7 across the 8 of their papers we have counts for
4 papers · 1 filter
Do Attention Heads Compete or Cooperate during Counting?
Pál Zsámboki, Ádám Fraknói, Máté Gedeon +2
We present an in-depth mechanistic interpretability analysis of training small transformers on an elementary task, counting, which is a crucial deductive step in many algorithms. I…
Towards Unbiased Exploration in Partial Label Learning
Zsolt Zombori, Agapi Rissaki, Kristóf Szabó +2
We consider learning a probabilistic classifier from partially-labelled supervision (inputs denoted with multiple possibilities) using standard neural architectures with a softmax…
Data-dependent Pruning to find the Winning Lottery Ticket
Dániel Lévai, Zsolt Zombori
The Lottery Ticket Hypothesis postulates that a freshly initialized neural network contains a small subnetwork that can be trained in isolation to achieve similar performance as th…
Gradient Regularization Improves Accuracy of Discriminative Models
Dániel Varga, Adrián Csiszárik, Zsolt Zombori
Regularizing the gradient norm of the output of a neural network with respect to its inputs is a powerful technique, rediscovered several times. This paper presents evidence that g…