281 citations
- Johannes Kepler University of LinzAT11 papers
- University of ViennaAT10 papers
- Innsbruck Medical UniversityAT9 papers
- University of Applied Sciences Technikum WienAT8 papers
- Austrian Academy of SciencesAT7 papers
- Sapienza University of RomeIT7 papers
- Universität InnsbruckAT7 papers
- Universidad de OviedoES4 papers
- Graz University of TechnologyAT3 papers
- Institute of Nuclear Physics, Polish Academy of SciencesPL3 papers
- Leibniz Institute for Solid State and Materials ResearchDE3 papers
- University of St.GallenCH3 papers
Showing 2024Show all
2 papers · 1 filter
cs.LG2024★ 3 cited
Efficient Neural Network Training via Subset Pretraining
Jan Spörer, Bernhard Bermeitinger, Tomas Hrycej +2
In training neural networks, it is common practice to use partial gradients computed over batches, mostly very small subsets of the training set. This approach is motivated by the…
cs.LG2024★ 1 cited
Reducing the Transformer Architecture to a Minimum
Bernhard Bermeitinger, Tomas Hrycej, Massimo Pavone +2
Transformers are a widespread and successful model architecture, particularly in Natural Language Processing (NLP) and Computer Vision (CV). The essential innovation of this archit…