8 citations · 10 across the 2 of their papers we have counts for
4 papers
Number of Attention Heads vs Number of Transformer-Encoders in Computer Vision
Tomas Hrycej, Bernhard Bermeitinger, Siegfried Handschuh
Determining an appropriate number of attention heads on one hand and the number of transformer-encoders, on the other hand, is an important choice for Computer Vision (CV) tasks us…
Training Neural Networks in Single vs Double Precision
Tomas Hrycej, Bernhard Bermeitinger, Siegfried Handschuh
The commitment to single-precision floating-point arithmetic is widespread in the deep learning community. To evaluate whether this commitment is justified, the influence of comput…
Representational Capacity of Deep Neural Networks -- A Computing Study
Bernhard Bermeitinger, Tomas Hrycej, Siegfried Handschuh
There is some theoretical evidence that deep neural networks with multiple hidden layers have a potential for more efficient representation of multidimensional mappings than shallo…
Singular Value Decomposition and Neural Networks
Bernhard Bermeitinger, Tomas Hrycej, Siegfried Handschuh
Singular Value Decomposition (SVD) constitutes a bridge between the linear algebra concepts and multi-layer neural networks---it is their linear analogy. Besides of this insight, i…