30 citations · 39 across the 6 of their papers we have counts for
5 papers · 1 filter
Minimal Ranks, Maximum Confidence: Parameter-efficient Uncertainty Quantification for LoRA
Patryk Marszałek, Klaudia Bałazy, Jacek Tabor +1
Low-Rank Adaptation (LoRA) enables parameter-efficient fine-tuning of large language models by decomposing weight updates into low-rank matrices, significantly reducing storage and…
LoRA-XS: Low-Rank Adaptation with Extremely Small Number of Parameters
Klaudia Bałazy, Mohammadreza Banaei, Karl Aberer +1
The growth of large language models underscores the need for parameter-efficient fine-tuning. Despite its popularity, LoRA encounters storage and computational challenges when depl…
r-softmax: Generalized Softmax with Controllable Sparsity Rate
Klaudia Bałazy, Łukasz Struski, Marek Śmieja +1
Nowadays artificial neural network models achieve remarkable results in many disciplines. Functions mapping the representation provided by the model to the probability distribution…
Zero Time Waste: Recycling Predictions in Early Exit Neural Networks
Maciej Wołczyk, Bartosz Wójcik, Klaudia Bałazy +4
The problem of reducing processing time of large deep learning models is a fundamental challenge in many real-world applications. Early exit methods strive towards this goal by att…
Finding the Optimal Network Depth in Classification Tasks
Bartosz Wójcik, Maciej Wołczyk, Klaudia Bałazy +1
We develop a fast end-to-end method for training lightweight neural networks using multiple classifier heads. By allowing the model to determine the importance of each head and rew…