20 citations · 63 across the 11 of their papers we have counts for
9 papers · 1 filter
NeuroTrails: Training with Dynamic Sparse Heads as the Key to Effective Ensembling
Bram Grooten, Farid Hasanov, Chenxiang Zhang +9
Model ensembles have long been a cornerstone for improving generalization and robustness in deep learning. However, their effectiveness often comes at the cost of substantial compu…
Unveiling the Power of Sparse Neural Networks for Feature Selection
Zahra Atashgahi, Tennison Liu, Mykola Pechenizkiy +3
Sparse Neural Networks (SNNs) have emerged as powerful tools for efficient feature selection. Leveraging the dynamic sparse training (DST) algorithms within SNNs has demonstrated p…
Adaptive Sparsity Level during Training for Efficient Time Series Forecasting with Transformers
Zahra Atashgahi, Mykola Pechenizkiy, Raymond Veldhuis +1
Efficient time series forecasting has become critical for real-world applications, particularly with deep neural networks (DNNs). Efficiency in DNNs can be achieved through sparse…
Where to Pay Attention in Sparse Training for Feature Selection?
Ghada Sokar, Zahra Atashgahi, Mykola Pechenizkiy +1
A new line of research for feature selection based on neural networks has recently emerged. Despite its superiority to classical methods, it requires many training iterations to co…
Memory-free Online Change-point Detection: A Novel Neural Network Approach
Zahra Atashgahi, Decebal Constantin Mocanu, Raymond Veldhuis +1
Change-point detection (CPD), which detects abrupt changes in the data distribution, is recognized as one of the most significant tasks in time series analysis. Despite the extensi…
Deep Ensembling with No Overhead for either Training or Testing: The All-Round Blessings of Dynamic Sparsity
Shiwei Liu, Tianlong Chen, Zahra Atashgahi +6
The success of deep ensembles on improving predictive performance, uncertainty estimation, and out-of-distribution robustness has been extensively studied in the machine learning l…
Sparse Training via Boosting Pruning Plasticity with Neuroregeneration
Shiwei Liu, Tianlong Chen, Xiaohan Chen +7
Works on lottery ticket hypothesis (LTH) and single-shot network pruning (SNIP) have raised a lot of attention currently on post-training pruning (iterative magnitude pruning), and…
Quick and Robust Feature Selection: the Strength of Energy-efficient Sparse Training for Autoencoders
Zahra Atashgahi, Ghada Sokar, Tim van der Lee +4
Major complications arise from the recent increase in the amount of high-dimensional data, including high computational costs and memory requirements. Feature selection, which iden…
Topological Insights into Sparse Neural Networks
Shiwei Liu, Tim Van der Lee, Anil Yaman +5
Sparse neural networks are effective approaches to reduce the resource requirements for the deployment of deep neural networks. Recently, the concept of adaptive sparse connectivit…