19 citations · 29 across the 6 of their papers we have counts for
4 papers · 1 filter
Adaptive Computation Modules: Granular Conditional Computation For Efficient Inference
Bartosz Wójcik, Alessio Devoto, Karol Pustelnik +2
While transformer models have been highly successful, they are computationally inefficient. We observe that for each layer, the full width of the layer may be needed only for a sma…
Exploiting Activation Sparsity with Dense to Dynamic-k Mixture-of-Experts Conversion
Filip Szatkowski, Bartosz Wójcik, Mikołaj Piórczyński +1
Transformer models can face practical limitations due to their high computational requirements. At the same time, such models exhibit significant activation sparsity, which can be…
SLOVA: Uncertainty Estimation Using Single Label One-Vs-All Classifier
Bartosz Wójcik, Jacek Grela, Marek Śmieja +2
Deep neural networks present impressive performance, yet they cannot reliably estimate their predictive confidence, limiting their applicability in high-risk domains. We show that…
Continual Learning with Guarantees via Weight Interval Constraints
Maciej Wołczyk, Karol J. Piczak, Bartosz Wójcik +5
We introduce a new training paradigm that enforces interval constraints on neural network parameter space to control forgetting. Contemporary Continual Learning (CL) methods focus…