720 citations
- University of Applied Sciences St. GallenCH12 papers
- Technical University of MunichDE11 papers
- ETH ZurichCH10 papers
- University of OxfordGB9 papers
- University of ZurichCH7 papers
- RWTH Aachen UniversityDE6 papers
- University of PassauDE6 papers
- Ludwig-Maximilians-Universität MünchenDE5 papers
- Sapienza University of RomeIT5 papers
- University of KasselDE5 papers
- Centre National de la Recherche ScientifiqueFR4 papers
- Imperial College LondonGB4 papers
16 papers · 1 filter
A Convexity-dependent Two-Phase Training Algorithm for Deep Neural Networks
Tomas Hrycej, Bernhard Bermeitinger, Massimo Pavone +2
The key task of machine learning is to minimize the loss function that measures the model fit to the training data. The numerical methods to do this efficiently depend on the prope…
Diffusion-Scheduled Denoising Autoencoders for Anomaly Detection in Tabular Data
Timur Sattarov, Marco Schreyer, Damian Borth
Anomaly detection in tabular data remains challenging due to complex feature interactions and the scarcity of anomalous examples. Denoising autoencoders rely on fixed-magnitude noi…
Efficient Neural Network Training via Subset Pretraining
Jan Spörer, Bernhard Bermeitinger, Tomas Hrycej +2
In training neural networks, it is common practice to use partial gradients computed over batches, mostly very small subsets of the training set. This approach is motivated by the…
Reducing the Transformer Architecture to a Minimum
Bernhard Bermeitinger, Tomas Hrycej, Massimo Pavone +2
Transformers are a widespread and successful model architecture, particularly in Natural Language Processing (NLP) and Computer Vision (CV). The essential innovation of this archit…
Continual Learning in the Presence of Repetition
Hamed Hemati, Lorenzo Pellegrini, Xiaotian Duan +11
Continual learning (CL) provides a framework for training models in ever-evolving environments. Although re-occurrence of previously seen objects or tasks is common in real-world p…
Make Deep Networks Shallow Again
Bernhard Bermeitinger, Tomas Hrycej, Siegfried Handschuh
Deep neural networks have a good success record and are thus viewed as the best architecture choice for complex applications. Their main shortcoming has been, for a long time, the…