4 papers
Reading the Finetuning Prior: Verbatim Content Recovery via Contrastive Decoding Diffing
Michał Brzozowski, Zuzanna Dubanowska, Enrico Cassano +1
Narrowly finetuned language models memorize implanted content verbatim, but auditing what a deployed model has been taught, without access to its weights or training data, remains…
Semantic Adapter Routing with Fine-Tuning Task Embeddings
Enrico Cassano, Michał Brzozowski, Paolo Mandica +2
Parameter-efficient fine-tuning (PEFT) has led to model ecosystems in which a single backbone is paired with many task-specialized adapters. Given such a library, routing aims to s…
GPart: End-to-End Isometric Fine-Tuning via Global Parameter Partitioning
Paolo Mandica, Michał Brzozowski, Zuzanna Dubanowska +1
Low-rank adaptation (LoRA) has become the dominant paradigm for parameter-efficient fine-tuning (PEFT) of large language models (LLMs). However, its bilinear structure introduces a…
Representation-based Broad Hallucination Detectors Fail to Generalize Out of Distribution
Zuzanna Dubanowska, Maciej Żelaszczyk, Michał Brzozowski +2
We critically assess the efficacy of the current SOTA in hallucination detection and find that its performance on the RAGTruth dataset is largely driven by a spurious correlation w…