7 papers · 1 filter
Benchmarking Debiasing Methods for LLM-based Parameter Estimates
Nicolas Audinet de Pieuchon, Adel Daoud, Connor T. Jerzak +2
Large language models (LLMs) offer an inexpensive yet powerful way to annotate text, but are often inconsistent when compared with experts. These errors can bias downstream estimat…
CUB: Benchmarking Context Utilisation Techniques for Language Models
Lovisa Hagström, Youna Kim, Haeun Yu +4
Incorporating external knowledge is crucial for knowledge-intensive tasks, such as question answering and fact checking. However, language models (LMs) may ignore relevant informat…
Language Model Re-rankers are Fooled by Lexical Similarities
Lovisa Hagström, Ercong Nie, Ruben Halifa +3
Language model (LM) re-rankers are used to refine retrieval results for retrieval-augmented generation (RAG). They are more expensive than lexical matching methods like BM25 but as…
Fact Recall, Heuristics or Pure Guesswork? Precise Interpretations of Language Models for Fact Completion
Denitsa Saynova, Lovisa Hagström, Moa Johansson +2
Language models (LMs) can make a correct prediction based on many possible signals in a prompt, not all corresponding to recall of factual associations. However, current interpreta…
What Happens to a Dataset Transformed by a Projection-based Concept Removal Method?
Richard Johansson
We investigate the behavior of methods that use linear projections to remove information about a concept from a language representation, and we consider the question of what happen…
Can Large Language Models (or Humans) Disentangle Text?
Nicolas Audinet de Pieuchon, Adel Daoud, Connor Thomas Jerzak +2
We investigate the potential of large language models (LLMs) to disentangle text variables--to remove the textual traces of an undesired forbidden variable in a task sometimes know…
The Effect of Scaling, Retrieval Augmentation and Form on the Factual Consistency of Language Models
Lovisa Hagström, Denitsa Saynova, Tobias Norlund +2
Large Language Models (LLMs) make natural interfaces to factual knowledge, but their usefulness is limited by their tendency to deliver inconsistent answers to semantically equival…