544 citations · 552 across the 8 of their papers we have counts for
4 papers · 1 filter
Beyond Masked Sparsity: SNACK Enables Truly Sparse Neural Networks on GPU
Jafar Badour, Maurice van Keulen, Elena Mocanu
Deep neural networks continue to grow in parameter count, driving up training and inference cost on GPUs. Sparse neural networks and Dynamic Sparse Training (DST) promise to reduce…
When Data Is Scarce: Scaling Sparse Language Models with Repeated Training
Boqian Wu, Qiao Xiao, Patrik Okanovic +6
Scaling laws for dense LLMs under infinite data are well explored, but how sparsity interacts with limited data is not. In this work, we study sparse training in data-constrained r…
Memory-Efficient LLM Training with Dynamic Sparsity: From Stability to Practical Scaling
Qiao Xiao, Boqian Wu, Patrik Okanovic +6
Dynamic Sparse Training (DST) offers a promising paradigm for improving the training and inference efficiency of deep neural networks; however, we find that in large language model…
Short Term Prediction of Parking Area states Using Real Time Data and Machine Learning Techniques
Jesper Provoost, Luc Wismans, Sander Van der Drift +2
Public road authorities and private mobility service providers need information derived from the current and predicted traffic states to act upon the daily urban system and its spa…