5 papers
Sequence Repetition Enhances Token Embeddings and Improves Sequence Labeling with Decoder-only Language Models
Matija Luka Kukić, Marko Čuljak, David Dukić +2
Modern language models (LMs) are trained in an autoregressive manner, conditioned only on the prefix. In contrast, sequence labeling (SL) tasks assign labels to each individual inp…
Out-of-Distribution Detection by Leveraging Between-Layer Transformation Smoothness
Fran Jelenić, Josip Jukić, Martin Tutek +2
Effective out-of-distribution (OOD) detection is crucial for reliable machine learning models, yet most current methods are limited in practical use due to requirements like access…
Parameter-Efficient Language Model Tuning with Active Learning in Low-Resource Settings
Josip Jukić, Jan Šnajder
Pre-trained language models (PLMs) have ignited a surge in demand for effective fine-tuning techniques, particularly in low-resource domains and languages. Active learning (AL), a…
Leveraging Open Information Extraction for More Robust Domain Transfer of Event Trigger Detection
David Dukić, Kiril Gashteovski, Goran Glavaš +1
Event detection is a crucial information extraction task in many domains, such as Wikipedia or news. The task typically relies on trigger detection (TD) -- identifying token spans…
On Dataset Transferability in Active Learning for Transformers
Fran Jelenić, Josip Jukić, Nina Drobac +1
Active learning (AL) aims to reduce labeling costs by querying the examples most beneficial for model learning. While the effectiveness of AL for fine-tuning transformer-based pre-…