28 citations · 151 across the 25 of their papers we have counts for
3 papers · 1 filter
Number of Attention Heads vs Number of Transformer-Encoders in Computer Vision
Tomas Hrycej, Bernhard Bermeitinger, Siegfried Handschuh
Determining an appropriate number of attention heads on one hand and the number of transformer-encoders, on the other hand, is an important choice for Computer Vision (CV) tasks us…
Training Neural Networks in Single vs Double Precision
Tomas Hrycej, Bernhard Bermeitinger, Siegfried Handschuh
The commitment to single-precision floating-point arithmetic is widespread in the deep learning community. To evaluate whether this commitment is justified, the influence of comput…
Uncovering More Shallow Heuristics: Probing the Natural Language Inference Capacities of Transformer-Based Pre-Trained Language Models Using Syllogistic Patterns
Reto Gubelmann, Siegfried Handschuh
In this article, we explore the shallow heuristics used by transformer-based pre-trained language models (PLMs) that are fine-tuned for natural language inference (NLI). To do so,…