13 papers
MACRO: Markov Chain Routing of Transformer Layers
Paweł Batorski, Abtin Pourhadi, Akylgali Aitaza +2
Standard Large Language Models (LLMs) execute layers sequentially. Dynamic layer routing, i.e. search for a different execution path through layers involving layer repetitions, ski…
GROM: Gradient-Free Rapid One-Shot Machine Unlearning
Paweł Batorski, Przemysław Spurek, Paul Swoboda
Machine unlearning has become a critical capability for safely removing specific, sensitive knowledge from large language models (LLMs). Current state-of-the-art approaches primari…
Spurious Prompts: Can Irrelevant Prompts Steer Large Language Models?
Pawel Batorski, Abtin Pourhadi, Jerzy Sarosiek +2
Large language models are highly sensitive to prompts, but this sensitivity is usually studied through task-relevant instructions, demonstrations, or reasoning cues. In this paper,…
Transcoda: End-to-End Zero-Shot Optical Music Recognition via Data-Centric Synthetic Training
Daniel Dratschuk, Paul Swoboda
Optical Music Recognition (OMR), the task of transcribing sheet music into a structured textual representation, is currently bottlenecked by a lack of large-scale, annotated datase…
PLR: Plackett-Luce for Reordering In-Context Learning Examples
Pawel Batorski, Paul Swoboda
In-context learning (ICL) adapts large language models by conditioning on a small set of ICL examples, avoiding costly parameter updates. Among other factors, performance is often…
PRL: Prompts from Reinforcement Learning
PaweÅ Batorski, Adrian Kosmala, Paul Swoboda
Effective prompt engineering remains a central challenge in fully harnessing the capabilities of LLMs. While well-designed prompts can dramatically enhance performance, crafting th…