12 papers
MACRO: Markov Chain Routing of Transformer Layers
Paweł Batorski, Abtin Pourhadi, Akylgali Aitaza +2
Standard Large Language Models (LLMs) execute layers sequentially. Dynamic layer routing, i.e. search for a different execution path through layers involving layer repetitions, ski…
GROM: Gradient-Free Rapid One-Shot Machine Unlearning
Paweł Batorski, Przemysław Spurek, Paul Swoboda
Machine unlearning has become a critical capability for safely removing specific, sensitive knowledge from large language models (LLMs). Current state-of-the-art approaches primari…
Spurious Prompts: Can Irrelevant Prompts Steer Large Language Models?
Pawel Batorski, Abtin Pourhadi, Jerzy Sarosiek +2
Large language models are highly sensitive to prompts, but this sensitivity is usually studied through task-relevant instructions, demonstrations, or reasoning cues. In this paper,…
PLR: Plackett-Luce for Reordering In-Context Learning Examples
Pawel Batorski, Paul Swoboda
In-context learning (ICL) adapts large language models by conditioning on a small set of ICL examples, avoiding costly parameter updates. Among other factors, performance is often…
PRL: Prompts from Reinforcement Learning
PaweÅ Batorski, Adrian Kosmala, Paul Swoboda
Effective prompt engineering remains a central challenge in fully harnessing the capabilities of LLMs. While well-designed prompts can dramatically enhance performance, crafting th…
PIAST: Rapid Prompting with In-context Augmentation for Scarce Training data
Pawel Batorski, Paul Swoboda
LLMs are highly sensitive to prompt design, but handcrafting effective prompts is difficult and often requires intricate crafting of few-shot examples. We propose a fast automatic…