5 papers
A Structural Theory of Position Bias in Transformers
Hanna Herasimchyk, Robin Labryga, Tomislav Prusina +1
Transformer models systematically favor certain token positions, yet the architectural origins of this position bias remain poorly understood. This bias is closely connected to the…
Multi-Label Plant Species Prediction with Metadata-Enhanced Multi-Head Vision Transformers
Hanna Herasimchyk, Robin Labryga, Tomislav Prusina
We present a multi-head vision transformer approach for multi-label plant species prediction in vegetation plot images, addressing the PlantCLEF 2025 challenge. The task involves t…
Information Preserving Line Search via Bayesian Optimization
Robin Labryga, Tomislav Prusina, Sören Laue
Line search is a fundamental part of iterative optimization methods for unconstrained and bound-constrained optimization problems to determine suitable step lengths that provide su…
Efficient Line Search Method Based on Regression and Uncertainty Quantification
Sören Laue, Tomislav Prusina
Unconstrained optimization problems are typically solved using iterative methods, which often depend on line search techniques to determine optimal step lengths in each iteration.…
Compressing Sentence Representation with maximum Coding Rate Reduction
Domagoj Ševerdija, Tomislav Prusina, Antonio Jovanović +3
In most natural language inference problems, sentence representation is needed for semantic retrieval tasks. In recent years, pre-trained large language models have been quite effe…